Sorting is Linear - also in Go!

Wrttien in Go

package main

/*
   Sorting a list of positive int64 values is linear, also in Go!
   Author: Harald Schilly 
   Based on some bash script I found somewhere on the internetz ...
*/

import "time"

func main() {
  var vals = []int64 { 55, 1, 9, 0, 31, 11, 90, 11 }

  var ret  = make(chan int64)
  var done = make(chan bool)

  // main loop, just a single for loop
  for _, v := range vals {
    go func(v int64) {
      // if it doesn't sort well, increase the 1e5
      time.Sleep(1e5 * v)
      ret <- v
    } (v)
  }

  // output iterates over all results in the ret channel
  go func() {
    for i := 0; i < len(vals); i++ {
      println(<-ret)
    }
    done <- true
  }()

  <-done
}

Java XML DOM Document creation

/**
 * Since if found some crap on the Internet, here a better example for creating an XML document in Java. 
 * 
 * Copyright: Harald Schilly
 * License: Apache 2.0
 */


package at.schilly.aldap2.ue12131415xml;


import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Map;


import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import javax.xml.transform.OutputKeys;
import javax.xml.transform.Transformer;
import javax.xml.transform.TransformerException;
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.dom.DOMSource;
import javax.xml.transform.stream.StreamResult;


import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;


/** @author harald schilly */
public class MapToXml {
  
  final private SimpleDateFormat dateformat = new SimpleDateFormat("yyyy-MM-dd");
  
  /** this method takes a Map of Strings to Strings and builds an xml document.
   * 
   * @param xmlfn
   *          xml output filename
   * @param map
   *          just a plain Map
   * @throws ParserConfigurationException
   * @throws FileNotFoundException
   * @throws TransformerException */
  public void write(final File xmlfn, final Map map)
      throws ParserConfigurationException, FileNotFoundException,
      TransformerException {
    // Docbuilder to create the xmldoc
    DocumentBuilderFactory docbuilderfactory = DocumentBuilderFactory.newInstance();
    DocumentBuilder docbuilder = docbuilderfactory.newDocumentBuilder();
    Document xmldoc = docbuilder.newDocument();
    
    // That's the root element
    Element root = xmldoc.createElement("map");
    
    // last saved element
    Element lastsaved = xmldoc.createElement("lastSavedBy");
    lastsaved.setAttribute("name", "Harald Schilly");
    lastsaved.setAttribute("date",
        dateformat.format(Calendar.getInstance().getTime()));
    root.appendChild(lastsaved);
    
    // iterate over map and add the entry/value elements
    for (Map.Entry mapentry : map.entrySet()) {
      Element entry = xmldoc.createElement("entry");
      entry.setAttribute("key", mapentry.getKey());
      
      Element value = xmldoc.createElement("value");
      Node node = xmldoc.createTextNode(mapentry.getValue());
      
      value.appendChild(node);
      entry.appendChild(value);
      root.appendChild(entry);
    }
    xmldoc.appendChild(root);
    
    // get a transformer with the given .dtd
    TransformerFactory transormerfactory = TransformerFactory.newInstance();
    Transformer transformer = transormerfactory.newTransformer();
    transformer.setOutputProperty(OutputKeys.DOCTYPE_SYSTEM, "map.dtd");
    transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
    transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no");
    transformer.setOutputProperty(OutputKeys.INDENT, "yes");
    
    // transform and write to a file
    // use a StringWriter() object to write to a string.
    FileOutputStream fos = new FileOutputStream(xmlfn);
    BufferedOutputStream bos = new BufferedOutputStream(fos);
    StreamResult result = new StreamResult(bos);
    DOMSource source = new DOMSource(xmldoc);
    transformer.transform(source, result);
  }
}

Anchors in Sage Notebooks

If you have a lengthy Sage Notebook and you want to quickly jump to a certain header or paragraph as a reference, use HTML anchors. They work like follows:

1. An <a name="anchorname"></a> tag must be inserted at the position you want to jump to.
2. Reference it via <a href="#anchorname">some text</a>.

After that, you can jump to the marked position just by clicking on the some text link.

To insert rich text and HTML, use the rich text editor accessible via <shift>+<click on blue insert bar>. There is an "HTML" icon. Alternatively, you can use the html() function:
i.e. html('<a name="test"></a>') ...

dotA Itemkauf Optimierung

Mein Mitbewohner ließ nicht locker,
spielte täglich dieses dotA.

Nach jedem Itemkauf fragte er sich,
ob sich das nicht besser machen lies.

Hier ein ZIMPL Skript zum optimieren,
bitte schreib es fertig, sonst ists zum genieren.

Flott sagt dir SCIP dann ganz korrekt,
wo du am besten dein Gold hinsteckst.




# zimpl model for optimizing buying dotA items
# copyright 2010: harald schilly
# license: CC-BY-NC-SA 3.0


# helden
set helden := { 1 to 5 };


# freie plätze zum einkaufen
param frei[helden] :=  <1> 6, <2> 4, <3> 4, <4> 2, <5> 6;


# welche items es gibt
set items := { 1 to 3} ;


# kosten pro item
param itemcost[items] := <1> 100, <2> 50, <3> 63;


# menge der fähigkeiten die jeder item pushen kann
set abilities := { 1 to 4 };


# was jeweils besser wird
param itempower[abilities * items] := 
  |   1,  2,   3 |
|1|   5,  1,   0 |
|2|   0,  1,   4 |
|3|   1,  1,   8 |
|4|   1,  4,  10 |;


# basiswert der zum maximum einer spalte addiert wird
# notwendig in der zielfunktion für die gewichtung
param baseval := 1;


# gesamtbuget an gold
param budget := 500;


# die variablenmatrix
var buy[helden * items] integer >= 0 <= 6;


# man kann nur so viel kaufen wie geld da ist
subto costs:
  sum <h,i> in helden * items: 
    buy[h,i] * itemcost[i] <= budget;


# man kann nur so viel kaufen wie der
# held noch freie plätze hat
subto maxperheld: 
  forall in helden:
    sum in items: buy[h, i] <= frei[h];


# ich will die fähigkeiten meines teams maximieren
maximize obj: sum <h,i,a> in helden * items * abilities:
  (itempower[a, i] / (baseval + max <a> in abilities: itempower[a,i])) * buy[h, i];

Compiling Sage on my Atom N270 Netbook

Here are some notes to myself regarding compiling Sage on my HP Mini 2140 with an Atom N270 CPU. I'm running Linux Ubuntu 9.10. This successfully compiles Sage 4.3 and 4.3.1 and might also work for later version.

Download

# using aria2, first get aria2
$ sudo apt-get install aria2

# go to a local directory and just use the .metalink link
$ aria2c http://server/path/sage-x.y.z.tar.metalink

# note, that aria2c doesn't stop when it has finished.
# it will start seeding the file to others via bittorrent.
# You can terminate this by hitting Ctrl-C


####

# or download via http/ftp from the download page

Verify

# If you think your download might have been corrupted, verify it:
$ aria2c -V http://server/path/sage-x.y.z.tar.metalink

Extract

# any local directory is fine

$ tar xf sage-x.y.z.tar

Prerequisites

# You need some tools to compile Sage:

$ sudo apt-get install build-essential m4\
readline libreadline-dev gfortran texlive

# read more: Installation Guide

Setup Build Environment

$ cd sage-x.y.z

# get rid of some environment variables, unless
# you know what you do (i.e. ccache, ...)
$ unset CC
$ unset CXX

# see README.txt if you need this
$ export SAGE_FAT_BINARY="yes"

# if you have gfortran library problems
# find your correct paths via $ locate gfortran

# setting these variables is necessary on Ubuntu 9.10
$ export SAGE_FORTRAN=/usr/bin/gfortran
$ export SAGE_FORTRAN_LIB=/usr/lib/libgfortran.so.3
# note: do not start over compilation if that problem happens,

# you have to remove and clean up everything first

# there are two cores in the CPU

# use both of them in parallel!
$ export MAKE="make -j2"

Start Compilation

# in a resource friendly mode

$ ionice -c 3 nice make

Testing and Packaging


If compilation didn't end with an error (otherwise: search, sage-support and sage-devel or irc chat)

# Test the entire beast (2 for 2 CPU cores):

$ ./sage -tp 2 devel/sage-main
# or
$ ./sage -testall
# once again, please report problems

If you want to build a binary distribution, upload it to us at sagemath.org or send it to a friend with a similar machine+system:

$ ./sage -bdist x.y.z-LinuxVersion;

On systems like Ubuntu, you can shrink the resulting archive much smaller using lzma compression. "trans-compress" it via:

$ zcat sage-x.y.z-...tar.gz | lzma -zv > sage-x.y.z-...tar.lzma

and to extract the tar.lzma later:

$ tar --lzma -xvf sage-x.y.z...tar.lzma

Links

• Installation Guide
• Please report successes or failures in our wiki

Maintaining a Mirror Network

Here are some notes about maintaining a mirror network. I'm talking about the Sage software, which is a nice open source mathematics program. Besides the source, there are many prebuilt binaries for various platforms. In total, for each release once a month, there have to move about 14GB in 27 files to 18 mirror websites on all continents of the world. It's important to have nearby mirrors, because the network connectivity might be weak in some areas. In Nov. 2009, the virtual box image for windows alone generated about 1.5 TB of traffic - all other binaries and source combined probably also 1.5 TB. Therefore, the mirror networks net efficiency is approximately a tenfold increases of the outgoing data volume.

To ensure that only those mirrors which are online and up-to-date are listed on the website, a small Python script checks a time-stamp file with a content-specific checksum. Only if it is correct and retrievable, the mirror is included. This check happens every 10 minutes, using Linux's cron mechanism.

Recently I enhanced this to visualize how the mirrors perform over time. This is especially interesting when a new release is mirrored out into the world. To see how this looks like, here is the graphic for the Sage 4.2.1 rollout:

Time starts at the bottom, you see it's around 14:00 GMT on Nov. 17th. The first working mirror is "UW", which is the master. The master mirror is excluded, once there is at least another one in North America online - as you can see that's SFU and Boston (Harvard). You can also see that it takes more than one day to sync with the mirrors. Besides that the transfer takes its time, they are probably scheduled to start sync only once a day, the master mirror rejects too many simultaneous connections, too many users download directly from the main server or there is simply some other timeout! Why are there two breaks? Well, not all binaries are ready at the same time and the timestamp is for the whole mirror, so that you do not access an outdated mirror. The broken line of Yandex is also a bit odd, but that's because they have multiple servers and seem to switch between. 

How does it look right now? Here is the website for the mirror log visualization. There is also a link to how it looked last week.

Beyond just retrieving a file through http or ftp you can use a metalink file. Through a client it enables downloading from all available mirrors simultaneously and the client selects a few but fastest mirrors automatically. It also checks consistency and downloads are resumable. That's the ideal solution if you have a weaker connection and need to stop a huge transfer for some time or it is just flaky. Hint: use aria2 or DownThemAll. 

Ok, at last some words about hosting a mirror. Two things are important: sync often to avoid wasting time when you could transfer data, but only sync once at a time ;) I've written how i did set up a mirror (the Boston one) in a MirrorNetwork wiki page.


In case the main website is down, you can find a mirror here.


Thank's to all our mirror hosters! Without them Sage would probably never reach their users and new mirrors are always welcome, especially in southern asia!

Popularity of Sage

This post is about Sage download statistics.

I try to track the actual download events and derive some information about them. This post is about the timerange June 1st until Aug 12th (today). The basic numbers are, that Sage is downloaded most often in the USA (35%), followed by Germany (7%), France (5%), UK, Italy, Spain, Canada, China, Brazil and Japan (4 to 2 %). On average, Sage was downloaded 150 times each day during the last semester, and a bit more than 100 times during summer break. I hope, numbers will go up again ;)

But that's rather boring and I tried to put the numbers in context. I focused on those countries with more than one million inhabitants and scaled the download numbers with the population size or with the number of internet users. To my surprise, the hit-list looks totally different now!

Download Top 20 weighted by population

Switzerland

Austria

New Zealand

Uruguay

United States

Norway

Slovenia

Denmark

Finland

Canada

Australia

Singapore

Germany

Netherlands

Spain

Israel

France

Czech Republic

Greece

I was somewhat surprised to see Switzerland, Austria and New Zealand and Uruguay on top! Concerning Uruguay, I know that they use Sage at University.

Download Top 20 weighted by "internet users"

Niger

Uruguay

Guatemala

Austria

Greece

Israel

Switzerland

Puerto Rico

Ethiopia

Mali

Portugal

Czech Republic

New Zealand

Ireland

Australia

Spain

Armenia

France

Denmark

For me, that's also rather interesting. Of course, those countries with only a few internet users are easily on top (Niger), but e.g. Guatemala has more than 1 million users and so far I have never heard of them using Sage. In this context, our top 3 from the pure download numbers are way down below: USA #23, Germany #21, France #19!

Now the important questions:

What does that say about Sage adoption? Market potential? Is Uruguay a maths country and the USA not? Does it depend on the average income, GDP, ... (Switzerland & Austria vs. Uruguay & Guatemala?!?!) ... or just statistical noise?

If you want to see the whole data: online Webpage, CSV, ODS

Update:

Here the same weighted top-20 lists for "Visits" (a session of one or more pages impressions on sagemath.org or an related website (mirror, wiki, etc.))

Visists weighted by Population:

Iceland

Austria

Martinique

Switzerland

Luxembourg

New Zealand

Slovenia

Norway

Canada

Denmark

Australia

Finland

USA

Netherlands

Germany

Spain

Sweden

Uruguay

United Kingdom

France

Visits weighted by internet users:

Martinique

Iceland

Austria

Switzerland

Uruguay

Greece

Guatemala

Israel

Luxembourg

Australia

Germany

Portugal

Slovenia

Spain

Denmark

New Zealand

Ireland

Finland

Hungary

Norway

France

USA

Canada

Puerto Rico

United Kingdom

Update: whole spreadsheet, also for visits: online Webpage, CSV, ODS

H