Convert several IPython Notebooks to one huge PDF

The IPython Notebook is a neat interactive document. It mixes headings, formatted text (markdown), formulas, images/pictures and interactive code. This is very neat for creating well-documented code examples.

To make ordinary publishing possible, nbconvert converts this interactive json-based file format to ordinary static HTML or LaTeX. The nbviewer website is a well known example for such a repository of HTML-notebooks.

Converting to LaTeX is also very handy, because then you can compile them to static PDF files -- e.g. think of a proper report or article.

So far I know, each input document is converted to only one output document. For one of my projects, I wanted to merge a whole bunch of IPython Notebooks to one (very large) PDF document! To achieve this, I've looked into the various *.tplx template file and started to build on top of them two new templates: partial and master. As the names suggest, partial is the template for each partial document, and master is essentially empty, but combines all partial documents in an orderly fashion.

You can see this here: github:haraldschilly/python-fuer-mathematiker

In particular:

• partial.tplx: This removes all the preamble and heading stuff and sets the document class to the wonderful subfiles LaTeX class.
• master.tplx: This template contains the default header definitions (inherited from IPython's base template), adds some tweaks of my own (font family, margins, title, and subfiles master class) and an embedded Python script to magically gather all documents!

How it works: First, all partial IPython Notebook files are converted to LaTeX using the partial template. Then, the master.tplx contains a small embedded Python script, which iterates through all partial LaTeX files. They are named numerically as --chaptername. This script inserts all those subdocuments and if there is an  increments in the major-number, it inserts a \chapter{...} heading.

To orchestrate all this, a usual Makefile checks for dependencies and finally uses latexmk with the --shell-escape switch (which is necessary to run the embedded Python). Works like a charm!

I hope this is helpful for some of you.

Mein Email an den VAP

Habe soeben dies an vap1@aon.at gesendet:

Mit Entsetzen muss ich heute lesen, dass österreichweit eine Blokadeinfrastruktur für Netzzensur errichtet wird. Ihr Mitwirken an diesem Akt der Unmündigmachung aller Österreicher_innen ist eine Skandal. Legen Sie mir bitte dar, welche Gesetze beim bloßen Download exakt verletzt werden und in welcher Relation dies zu der angeprangerten Bundesbürger_innenbevormundung steht. Die alten Wirtschaftsmodelle haben ausgedient und Ihr VAP Verein ist eine reine Interessensvertreung für große, längst überholte internationale Konzerne und im Beamtenschimmel verfilzte österreichische Organisationen. Ich hoffe Sie bekommen in den nächsten Wochen und Monaten reichlich Gegenwind zu spüren. Willkommen im 21-ten Jahrhundert!

In Vollendung,
ein Bundesbürger

PS:

Als Wochenendlektüre empfehle ich den Barbara Streisand Effekt:
https://en.wikipedia.org/wiki/Streisand_effect

Netflix in Ubuntu - part 2347 of \infinity

Today, Netflix started in Austria. Well, I thought, give it a try. Too bad, it's really painful in my Linux box. So, if you also have Ubuntu 14.10:

1. Get Chrome Beta 38 or higher
2. Update libnss3 via running this in your terminal:
   
   for pkg in libnss3_3.17-1ubuntu1_amd64.deb libnss3-nssdb_3.17-1ubuntu1_all.deb libnss3-1d_3.17-1ubuntu1_amd64.deb
   do
      wget http://archive.ubuntu.com/ubuntu/pool/main/n/nss/${pkg}
   doneThen install it:
   
   $ sudo dpkg -i libnss*.deb
3. Install the User-Agent Switcher for Chrome
   In its options (right-click on the icon) you have to add a new entry (besides the "String", nothing else really matters):
   
   
   • Name: Netflix
   • String: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2114.2 Safari/537.36
   • Group: keep it empty
   • Indicator: FLX (for Net-Flix)
   
   After that, set it to be active on netflix.com automatically: On the left, in "Permanent Spoof List"...
4. Restart chrome, ... cross your fingers ... and enjoy.
5. Edit: you also have to set the profile properties in netflix to HTML5 (though, I think it still works without doing anything, because there is no silverlight anyways)

New combinatorial designs in Sage - by Nathann Cohen

This is a guest post by Nathann Cohen. 

---

New combinatorial designs in Sage

Below, these graphs are a decomposition of a $K_{13}$ (i.e. the complete graph on 13 points) into copies of $K_4$. Pick two vertices you like: they appear exactly once together in one of the $K_4$.

The second graph shows a decomposition of a $K_{4,4,4}$ (i.e. the complete multipartite graph on $4\times 3$ points) into copies of $K_3$. Pick two vertices you like from different groups: they appear exactly once together in one of the $K_4$.

Sage has gotten quite good at building such decompositions (a specific kind of combinatorial designs) when they exist. This post is about them.

The first object belongs to a family called Balanced Incomplete Block Designs (or $(n,k)$-BIBD), which are defined as "a collection $\mathcal S$ of sets, all of them with size $k$ (here $k=4$), such that any pair of points of a set $X$ with $|X|=n$ (here $n=13$) appears in exactly one set of $\mathcal S$".

The second belongs to the family of Transversal Designs (or $TD(k,n)$) which have a similar definition: consider a set $X$ containing $k$ groups (here $k=3$) of $n$ vertices (here $n=4$). A collection $\mathcal S$ of sets, each of which contains one point from each group, is a $TD(k,n)$ if any two points from different groups appear together in exactly one set of $\mathcal S$.

The main problem with combinatorial designs is to know where they exist. And that is not obvious. Sage does what it can on about that:

• If you want it to build a $(14,4)$-BIBD, it will tell you that none exists.
• If you want it to build a $(16,4)$-BIBD, it will tell you that one exists.
• If you want it to build a $(51,6)$-BIBD, it will tell you that it just not know if there is one (and nobody knows better at the moment)

Examples here:

sage: designs.balanced_incomplete_block_design(14,4,existence=True)
False
sage: designs.balanced_incomplete_block_design(16,4,existence=True)
True
sage: designs.balanced_incomplete_block_design(51,6,existence=True)
Unknown

For a developer (and design lover), the game consists in teaching Sage how to build all combinatorial designs that appear in some research paper. For BIBD as well as for Transversal Designs, on which a LOT of sweat was spent these last months.

For Transversal designs the game is a bit different, as we know that a $TD(k-1,n)$ exists whenever a $TD(k,n)$ exists. Thus, the game consists in finding the largest integer $k_n$ such that a $TD(k_n,n)$ exists. This game is hardly new, and hardly straightforward: In the Handbook of Combinatorial Designs, one can find the table of such $k$ up to $n=10000$ (see here).

The good thing about Sage is that it does not just claim that such a design exists: it also builds it, and there is no better existence proof than that (it is very very quick to check that a combinatorial design is valid). The other good thing is that there is no common database for such data (the Handbook is not updated/printed every night), and that by teaching Sage all new designs found by researchers we build such a database. And it already contains designs that were not known when the Handbook was printed.

Finally, the other other good thing about Sage is that it will soon be able to tell you where those designs come from. Indeed, the most powerful results in the field of Transversal Designs are of the shape "If there exists a $TD(k_1,n_1)$, and a $TD(k_2,n_2)$, ..., and a $TD(k_c,n_c)$, then you can combine them all to obtain a $TD(k,n)$ with $k=k(k_1,...,k_c)$ and $n=n(n_1,...,n_c)$". And it is never very clear how to inverse these functions: if you want to build a $TD(k,n)$, which integers $k_1,...,k_c,n_1,...,n_c$ should you pick ?

Sage knows. It must know it, in order to build these designs anyway. And you can find that data inside. And soon, we will teach it to give you the bibliographical references of the papers in which you can find the right construction to produce the $TD(k,n)$ that you want. And we will provide the right parameters. And the world will be at peace.

A couple of things before we part:

• Transversal Designs (TD), Orthogonal Arrays (OA), and Mutually Orthogonal Latin Squares (MOLS) are all equivalent objects.
• We write a LOT of Transversal Designs code these days, so expect all this to improve very fast.
• You can learn what Sage knows of combinatorial designs right here.

Finally, there are far too many combinatorial designs for one man to learn. If you love combinatorial designs, come join us: Vincent Delecroix, you, and I have code to write together. And if you know a related mathematical results that Sage ignores, come tell us: we could not have gone so far without the mathematical knowledge of Julian Abel. And Sage does not know everything yet.

Have fuuuuuuuuuuuuuuuuuuun !

Nathann

Blüte des Lebens - Georg Kostron

Musik-kritische Ausführungen im Staccato.

Vorweg: dieser Text wird in Kürze ins Lateinische übersetzt.

"Blüte des Lebens" ist eine stimmgewaltige Kraftkomposition. Ausgetüftelte Klänge begleiten diversiv zusammengetragene Klangschnipsel und garnieren - begleitet durch zeitkritischen Text - den ausgeprägten Hang zum liebevollen Detail. Das Album thematisiert an dessen Höhepunkt das Geschraube und Gedrehen an den Knöpfen der absoluten Machtkontrolle in der zentralen Schaltzentrale. Kurz: der tüftelnde Monarch seines unendlich großen Reichs der Effektkiste ist politisch.

Song 01. Raumschiff Maria

Ein routinierter Klick auf das nach rechts blickende Abspieldreieck, es wummert kurz - Dissonanz - zu leise? - kurzes Quietschen und sofort nach der Landung ist klar: dieses Album ist eingekochter Grüntee statt verdünntem Fruchtsaft. Der 1/4 Beat klopft hart ins Ohr, ist verwirrt ob des Textes und der vorsichtige Mitteleuropäer dreht ab Minute 2:30 sicherheitshalber etwas leiser. Ist das etwas zum Beten?

Song 02. Rostiges Rösslein

Versöhnlicher, Chorgesang, Melodie - zurücklehnen. Aber spätestens das Textfragment "Geld arbeitet - nicht" lässt die untertönig bittere Note anklingen. "Zügellos", "Sporen", "Peitsche" und todesbleiche Reiter mahnen den Hörer.

Song 03. Der kleine Tiger

Katzen möchten im monotonen Alltag eingelullt, sicher in der Box hockend, den Tag absitzen. Textlastiges Crescendo, maximal aufgedrehte Stimmen und songstrukturierte Abwechslung konfrontieren sie. Das Album schafft sich Raum und dehnt ... aus. Inneres Unbehagen, Stress und Unentspannung statt schmeichelnder Wohlfühlklänge.

Song 04. Lick My Face (edit)

Besuch einer altbekannten Zeitgenössin aus der "Sophia I" Phase. Musikalisch geben sich Elektronik, Instrument und die menschliche Mundschallöffnung die Hand. Sie zerren sdich gegeneinander in ihre jeweiligen Ecken, kurz kippt es in eine Ecke, nur um später explosiv zurückzuschnalzen.

Verbraucherhinweis: Spätestens an dieser Stelle wird eine erste Verschnaufpause empfohlen.

Song 05. Alle Welt

Vermeintlich poppig geht es weiter, aber der Refrain bestätigt: dies täuscht. Ein Abgesang auf das kollektive Weltbild. Inzwischen passen die sperrigen Klänge problemlos in den weit aufgemachten Raum. Es knarzt.

Song 06. Knapp vor Schaltraum

Herrlich! Das intellektuell stimulierende Latein dämpft die aufgewühlten animalistischen Emotionen, es trägt, man spürt: das Zentrum kommt für Wahr näher und man steht im Auge des Taifuns. Avantgardistische Elektronik, dann eine Orgel, ganz von der Ferne kanonenschussartige Paukenschläge.

Song 07. Süßes Kind

Fast fließend landet der Hörer in einem Liebeslied? Wer wird hier besungen? Die aufopfernde Schlusssteigerung dreht nochmals schnell am Sinn, erzeugt Verwirrung, die Entspannung ist schnell vorbei und schon ist 08 bereit für die Bühne.

Song 08. Schaltraum (edit)

Bitte Anschnallen. Treibender, hämmernder, minimalistischer Druck. Die Beats tänzeln mit sich selbst im trockenen Vakuum. Ein technokratisch editiertes Interludum entzieht jegliche Humanität, die elektrische Steuerung übernimmt das Ruder. Fast spirituelle Songskizzen beenden diesen Raubzug am Verstand. Wer hat ihn geraubt?

Song 09. Beistrich

Ein versöhnlich-humoristischer Aufgesang an die jugendliche Interpunktionskultur des Web 2.0.

Song 10. Wunderschön (edit)

Hitverdächtige Durtöne, Chöre, Mitsingmelodie, Geige, hin-und-her: heftig illussioniert, warm eingewickelt, was ist da los? Relativierende Schlagwörter gravitieren den Hörer wieder auf den rauhen Betonboden der Realität. Diese Blendung ist eine erfolgreiche.

Song 11. Küss den Tango

Der vielleicht treibendste Song des Albums. Kurzweilige Strukturzerlegungen und eingeworfene Schrägtöne bringen die schon gewohnte Unsicherheit wieder zurück. Auf-und-ab gesungene "Ah"s schrubben die bereits geröteten Großhirnfurchen wieder sauber. Es riecht nach Schweiß und Reinigungsmittel.

Song 12. Hans Tanz

Zurücklehnende Melodien, breit, kaugummiartig. Ein Akkordeon, etwas Echo, und diverse Aufforderungen an Hans verhallen im Hintergrund. Nachdenklichkeit macht sich breit. Ein gänsehautgarantierender Schlusschor.

Song 13. Der A

Nicht die Spur einer Scheu vor der Zahl 13. Es soll einfach weiter-so gehen, halt, ja, einfach weiter ... so. Hilflos gefangen im musikalischen Spinnennetz dieses Albums spannt sich ein weiterer Radialfaden in Richtung klassischem Clubsound. Tanz endlich. Im Kreis. Spätestens seit Snoweden wissen wir, dass sich auch Drohnen ständig im Kreis herum drehen.

Song 14. Tanzboutique

Der letzte Song. Ist Steigerung überhaupt noch möglich? Ein Schrei, Auftakt, 3, 2, 1. Was passiert hier? Ein sauber auskomponierter Song im deutsch-lateinischen Zwiegespräch wirft mit mehrdeutigen Fragmenten um sich. Dem Hörer wird ein kleine geheime Tür gezeigt: es ist der Notausgang in die Wirklichkeit.

Bonustrack.

Wer ein paar Minuten warten kann, wird mit einem acapellaartigem Songfragment belohnt. Es endet offen, ein Vorgeschmack auf einen Teil 2?

Abschlussbeurteilung:

9 von 10 Aes Grave.

Links:

f/undSeinManager

Sage GSoC 2014 Projects

This year, Sage is happy to announce that it will be running five Google Summer of Code projects. Welcome everyone to the Sage community and all the best to your summer filled with exiting projects ;-)

EDIT: lmonade related projects are listed here.

Nikhil Peter Raj (Volker Braun):

Improvements to the Sage Android App

This project aims to improve upon the existing app by improving both internal structure and external user experience, the former by replacing and redoing certain modules of the app (JSON Parsing, HTTP Requests and Exception Handling) and the latter by redesigning the UI so it conforms to the latest standards and specifications of Android Design, as well as adding extra features to the code input such as Syntax Highlighting and options to record and store custom inserts.

Amit Jamadagni (Miguel Angel Marco-Buzunariz)

Knot theory implementation

The project mainly deals with the implementation of various links, knots presentations and calculating various related invariants. The main aim of the project is to achieve: Conversion between different representations of Knots and Links to mention a few: Gauss code to Braid Word, Braid word to DT code, Gauss code to DT code. Implementation of various invariants: Alexander Polynomial, Conway Polynomial. Stiefert Matrix, Jones Polynomial

Jayant Apte (Stefan van Zwam)

Efficient class-specific membership checks, extensions and visualization 

First goal of this proposal is to implement efficient testing of a matroid for membership of various classes and to perform single element extensions of matroids that exploit the knowledge that a given matroid belongs to a certain class, to have lower complexity. We also propose implementation of visualization techniques and an application of matroid theory in form of enumeration of network codes for a multisource network coding problem which is one of the timely problems in information theory.

Simon Spicer (William Stein, Jonathan Bober)

Efficient exact analytic rank computation for elliptic curves in Sage

My project would be to implement functionality in Sage to compute the analytic rank of an elliptic curve exactly modulo standard conjectures, with better scaling than existing analytic rank estimation methods. This would necessitate both writing (significant amounts of) new Python and Cython code, as well as porting existing code into the Sage codebase.

Daniel Bell (Ivan Andrus)

iSage - improving the Sage iOS apps

There are several aspects of the Sage iOS apps that needs to be refreshed, improved or implemented. I propose to develop the ability to interact with several Sage services from the app, as well refresh the user interface for the app.

My first Time-Lapse Experiment

Well, my new camera (Panasonic DMC-GX7) has two features I've never had before: stop-motion and time-lapse. The only difference seems to be: the minimum time between the shots is either 1 second or 1 minute - and if there is an "open end" or a fixed number of pictures.

Searching around online, the most important thing to do is to take the shots in manual mode. The reason is, that you do not want to have this flickering. So well, that's easy to do in manual mode: exposure, aperture, ISO and of course the focus. What I didn't read (or didn't notice), but should have done is to also fix the white-balancing! So, in case you want to try this, set it the white balancing to manual.

One cool thing of doing this in stock motion on my camera is, that it automatically converts the images to a movie. Only for one sequence I had to do it on my computer, because the camera wasn't responsive or I've confused it. So, in case you are on a Ubuntu Linux like I am, here is what to do: First, don't bother with ffmpeg or avconv: they are too old and do not have the flag "-start_image" to set the first number of the counter. This sucks!

So instead: mencoder:
Start by moving all the images in a separate directory, then the "P*.JPG" regex pattern matches all your images in alphabetical order. Easy.

$ mencoder -nosound -ovc lavc -lavcopts vcodec=mpeg4:mbd=2:trell:autoaspect:vqscale=3 -vf scale=1920:1080 -mf type=jpeg:fps=5 'mf://P*.JPG' -o OUTPUT.avi
=> see update below

If you have better ideas for the command-line, please post it. fps=5 is for the number of frames per second. I think the quality should be set higher, but no idea which of the switches does this in a sane way.

Now enjoy these three YouTube videos from Vienna (Parlament and City-Hall):

Update:

There is indeed a much better mencoder setting, using x264:

$ mencoder -nosound 'mf://P*.JPG' -mf fps=10:type=jpeg -ovc x264 -x264encopts preset=slow:tune=animation:crf=20 -vf scale=1440:1080 -o time-lapse.avi

Options (for more, check the manpage)

• fps=10: 10 pix/second. 5 is also ok, 20 if you have really a lot of pics.
• tune=animation: sounds reasonable, there is also "film"
• crf=20: recommended switch for setting the quality, could also be 10 or 20, ...
• -vf scale: that could be something different, depends on what you have. (960x720)