Merge branch 'localization' into 'main'

Localization

See merge request !6

0_Installation_de.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "31ac33c0-18a9-4cfa-8e04-3ee1a0c909eb",
+   "metadata": {},
+   "source": [
+    "# Installation und Einrichtung\n",
+    "Bitte befolge die in README.md beschriebenen Schritte 1-3, um Julia herunterzuladen und zu installieren, und das Notebook 0_Installation.ipynb in einen Jupyter-notebook zu öffnen."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ff3c6951-2bc8-42b9-b2f4-acbb473300c6",
+   "metadata": {},
+   "source": [
+    "## Informationen zu Notebooks\n",
+    "\n",
+    "Notebooks sind Werkzeuge, um interaktiven Code auszuführen und die Ausgabe anzuzeigen. Sie sind sehr praktisch, um mit Code zu experimentieren und eine Programmiersprache neu zu lernen.\n",
+    "\n",
+    "Die Julia-Befehle werden in einer separaten Julia-Instanz im Hintergrund ausgeführt, der Browser ermöglicht die Interaktion mit den Notebooks (das Erstellen neuer Notebooks, das Ausführen und Modifizieren von ihnen). Beachte dass der Browser-Editor nicht mehr funktioniert wenn das Backend-Programm geschlossen wird.\n",
+    "\n",
+    "Wenn jedoch der Browser geschlossen wird, läuft das Backend weiter. Du kannst darauf zugreifen, indem Du die Adresse öffnest:\n",
+    "[http://localhost:8888](http://localhost:8888)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c974b7c0-119c-4223-ad80-dce5a3ceead3",
+   "metadata": {},
+   "source": [
+    "### Notebooks\n",
+    "\n",
+    "Notebooks sind Dateien, die durch die Erweiterung `.ipynb` gekennzeichnet sind. Mit dem interaktiven Datei-Explorer kannst Du sie öffnen.\n",
+    "\n",
+    "Die Notebooks enthalten Code, die Ausgabe des Codes und können zusätzliche Kommentare enthalten (wie diese hier). Das gesamte Material des Kurses enthält noch nicht die Ausgabe des Codes, Du erhälst sie, indem Du den Code zuerst ausführst.\n",
+    "\n",
+    "Die Notebooks werden automatisch alle paar Minuten gespeichert, um jedoch sicherzustellen, dass sie gespeichert sind, kannst Du \"Strg + S\" drücken.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8c98e5f5-5a92-415a-b9ec-865972d45b6f",
+   "metadata": {},
+   "source": [
+    "### Zellen\n",
+    "\n",
+    "Notebooks bestehen aus **Zellen**. Es gibt zwei Hauptarten: **Code** und **Markdown**. Dies ist eine Markdown-Zelle und wird verwendet, um Text auf eine formatierte Weise anzuzeigen.\n",
+    "\n",
+    "Die nächste ist eine Code-Zelle und wird verwendet, um Code auszuführen und dessen Ausgabe anzuzeigen. Um sie auszuführen, klicken Sie darauf und drücken Sie Shift + Enter."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8074a659-e100-49db-b83c-f0738b25cf28",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "println(\"Hello World!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "192dac00-072c-4699-bd06-9af6ae4e3055",
+   "metadata": {},
+   "source": [
+    "Wenn Du auf eine Code-Zelle klickst, kannst Du auch deren Inhalt ändern. Um eine Markdown-Zelle zu ändern, musst Du darauf doppelklicken (dies ist jedoch für diesen Kurs nicht erforderlich).\n",
+    "\n",
+    "Es ist auch möglich, eine Code-Zelle in eine Markdown-Zelle oder umgekehrt zu ändern. Gehe dazu im oberen Menü auf Ausführen, Zellentyp.\n",
+    "\n",
+    "Schließlich können neue Zellen hinzugefügt werden, indem Du das `+`-Symbol im oberen Menü verwendest.\n",
+    "\n",
+    "Jedes Mal, wenn es eine Übung für den Leser gibt, haben wir eine leere Code-Zelle hinzugefügt."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e7730114-5064-46d1-b724-e92277ff2f90",
+   "metadata": {},
+   "source": [
+    "## Überprüfen, ob alles funktioniert\n",
+    "\n",
+    "Um zu überprüfen, ob die Installation erfolgreich war, führe die folgende Zelle aus."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7bb04662",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "using Plots\n",
+    "\n",
+    "p = plot(1, xlim=(0, 3π), ylim=(-1.5, 1.5), title=\"Sine\", marker=2)\n",
+    "N = 100\n",
+    "@gif for i=1:N\n",
+    "    x = (i-1) * 3π / N\n",
+    "    push!(p, x, sin(x))\n",
+    "end"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c7d2870e",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "julia 1.9.3",
+   "language": "julia",
+   "name": "julia-1.9"
+  },
+  "language_info": {
+   "file_extension": ".jl",
+   "mimetype": "application/julia",
+   "name": "julia"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

0_Installation_en.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "16b28ead",
+   "metadata": {},
+   "source": [
+    "# Installation and Setup\n",
+    "Please follow the steps 1-3 outlined in README.md to download and install Julia, and to open the notebook 0_Installation.ipynb in a Jupyter notebook. "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9161160b-87ff-4b12-bd5a-bca54228672d",
+   "metadata": {},
+   "source": [
+    "## Information about notebooks\n",
+    "\n",
+    "Notebooks are tools to interactively run code and display output. They are quite convenient for experimenting with code.\n",
+    "\n",
+    "The julia commands are run in a backend julia instance, the browser let's you interact with the notebooks (create new ones, run them, modify them). If the backend program is closed, the browser editor will stop working.\n",
+    "\n",
+    "However, if the browser is closed, the backend will keep running. You can get back to it by opening the address:\n",
+    "[http://localhost:8888](http://localhost:8888)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a915cd2f-dc38-4679-8d79-c2204e2073cf",
+   "metadata": {},
+   "source": [
+    "### Notebooks\n",
+    "\n",
+    "Notebooks are files characterized by the extension `.ipynb`. Using the interactive file explorer you can open them.\n",
+    "\n",
+    "The notebooks contain code, the output of the code, and can contain additional comments (like these ones). All the material of the course does not contain yet the output of the code, you will get it by running the code first.\n",
+    "\n",
+    "The notebooks are automatically saved every now and then, however to make sure they are saved, you can press \"Ctrl + S\"."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "93606623-16a4-4263-9906-a79dd3fae22f",
+   "metadata": {},
+   "source": [
+    "### Cells\n",
+    "\n",
+    "Notebooks are made up of **cells**. There are two main types: **code** and **markdown**. This is a markdown cell, it is used to display text in a formatted way.\n",
+    "\n",
+    "The next is a code cell, it is used to run code and display its output. To run it, click on it and press \"Shift + Enter\"."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "8074a659-e100-49db-b83c-f0738b25cf28",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "println(\"Hello World!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5fc38ba1-65d0-490f-8c06-4136a7ff1372",
+   "metadata": {},
+   "source": [
+    "When you click on a code cell, you can also modify its content.\n",
+    "To modify a markdown cell, you need to double click on it (however this is not required by this course).\n",
+    "\n",
+    "It is also possible to transform a code cell in a markdown cell, or the other way around. To do so, go in the top menu, Run, Cell Type.\n",
+    "\n",
+    "Finally, new cells can be added by using the `+` symbol in the top menu.\n",
+    "\n",
+    "Every time there is an exercise left to the reader, we added an empty code cell."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d3d75953",
+   "metadata": {},
+   "source": [
+    "## Verifying that everything works\n",
+    "\n",
+    "To check if the installation was successfull, run the following cell."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7bb04662",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "using Plots\n",
+    "\n",
+    "p = plot(1, xlim=(0, 3π), ylim=(-1.5, 1.5), title=\"Sine\", marker=2)\n",
+    "N = 100\n",
+    "@gif for i=1:N\n",
+    "    x = (i-1) * 3π / N\n",
+    "    push!(p, x, sin(x))\n",
+    "end"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "c7d2870e",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "julia 1.9.3",
+   "language": "julia",
+   "name": "julia-1.9"
+  },
+  "language_info": {
+   "file_extension": ".jl",
+   "mimetype": "application/julia",
+   "name": "julia"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

src/localize.py

+#!/usr/bin/python3
+
+import copy
+import json
+import os
+import sys
+
+import langid
+
+
+def detect_language(text):
+    return langid.classify(text)
+
+
+def load(path):
+    with open(path, encoding="utf-8") as file:
+        notebook = json.load(file)
+    return notebook
+
+
+def human_intervention(language, probability, text):
+    print(
+        f"I classified this text:\n`{text}`\nas {language} with a non normalized probability of {probability}.\n"
+    )
+    while True:
+        print(
+            "Can you help me identifying it? write one among 'en', 'de', 'eq' (for equations), then press enter"
+        )
+        language = input()
+        if language in ["en", "de", "eq"]:
+            print("\n\n")
+            return language
+
+
+def write(path, notebook):
+    with open(path, "w", encoding="utf-8") as file:
+        json.dump(notebook, file, indent=4)
+
+
+def assign_languages(notebook):
+    cells = notebook["cells"]
+
+    for i, cell in enumerate(cells):
+        if cell["cell_type"] == "markdown":
+            text = str(cell["source"]).strip("\n# []'")
+            language, probability = detect_language(text)
+
+            if probability >= -10 or language not in ["en", "de"]:
+                # langid is doing something strange, we need to intervene
+                language = human_intervention(language, probability, text)
+
+            notebook["cells"][i]["metadata"]["natural_language"] = language
+
+
+def select_language(notebook, language):
+    cells = notebook["cells"]
+
+    selected_cells = []
+    for cell in cells:
+        if cell["cell_type"] == "markdown":
+            if cell["metadata"]["natural_language"] == language:
+                selected_cells.append(cell)
+            elif cell["metadata"]["natural_language"] == "eq":
+                selected_cells.append(cell)
+        else:
+            selected_cells.append(cell)
+
+    new_notebook = copy.deepcopy(notebook)
+    new_notebook["cells"] = selected_cells
+
+    return new_notebook
+
+
+if __name__ == "__main__":
+    PATH = sys.argv[1]
+    BASENAME = os.path.splitext(os.path.basename(PATH))[0]
+
+    NOTEBOOK = load(PATH)
+    assign_languages(NOTEBOOK)
+
+    NOTEBOOK_EN = select_language(NOTEBOOK, "en")
+    NOTEBOOK_DE = select_language(NOTEBOOK, "de")
+
+    write(f"{BASENAME}_en.ipynb", NOTEBOOK_EN)
+    write(f"{BASENAME}_de.ipynb", NOTEBOOK_DE)