How to link Intel MKL lapack

This article summarizes how to link existing C++ code with Intel MKL LAPACK without modifying a single line of the original source code that was written with standard LAPACK libraries and compiled by g++.

Intel MKL comprises so many things and for my purpose. In my case, I only want to know if MKL can provide better performance compared to GotoBLAS in my application. So I don't want to change any part of my code and just want to replace GotoBLAS with MKL. This article is written for pure newbies to Intel MKL and is not intended for learning MKL properly.

The task of replacing LAPACK with MKL turns out to be very simple: I used to link my code with GotoBLAS by flag -lgoto2 or standard LAPACK by -llapack. For MKL, this website helps a lot:

http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

After filling my platform information (GNU C/C++ and libgomp), the following flags are suggested:

 -L$MKLROOT/lib/intel64 -lmkl_rt -ldl -lpthread -lm

I replace -libgoto2 with the above flags, recompile my code, and then my code runs smoothly with MKL (and much faster than GotoBLAS for multithreaded cases but slower when single threaded).

This article here summarizes how to control the threads that MKL for the LAPACK part

http://software.intel.com/sites/products/documentation/hpc/mkl/lin/MKL_UG_managing_performance/Using_the_Intel_MKL_Parallelism.htm

In short, MKL automatically uses the number of physical cores as default number of threads. Hyperthreading is not considered as very little juice can be squeezed out when highly optimized and efficient code such MKL is used. If you want to control the number of threads, several global variables can be used and described clearly here:

http://software.intel.com/sites/products/documentation/hpc/mkl/lin/MKL_UG_managing_performance/Using_Additional_Threading_Control.htm

The global variables are independent from OpenMP counterparts. In my case, for example, when I want to use two threads only to solver my linear system Ax=b, I enter the following command in bash:

export MKL_NUM_THREADS=2

before I run my executable. Different performance being observed before and after setting up the global variable confirms the success of my attempt.

Yalmip with Mosek in SOS optimization

This post describes how to make Mosek work smoothly with Yalmip in SOS program. The following  guide is for academic use on Linux Mint 15 (Ubuntu 13.04) with Matlab R2012a, Mosek 7.0.0.65, and Yalmip R20130405.

Installation of Mosek

You simply download it from Mosek.com, untar the downloaded file directly under your home folder, apply for the academic license, and place the license file directly under the mosek folder. The mosek folder and license place are rigid. Changing them may make mosek not function properly.

Mosek for Linux is precompiled and only shipped in .a and .so files. So you won't encounter make hiccups and you can't easily decipher what is going on within. You may either have it work like a breeze or not work at all. And there is nothing you can do if in the second case. (Or few things but anyway not much like cracking the code because the source is not shipped.)

If you want to use mosek in command line, edit your .bashrc with this additional line:

export PATH="$PATH:$HOME/mosek/7/tools/platform/linux64x86/bin"

Then "source .bashrc" or restart your terminal. You can try type "mosek" in your new open-up terminal to see if the path has been set up correctly.

If you want to use mosek in Matlab only, set up mosek path in your Matlab search path:

/home/youraccount/mosek/7/toolbox/r2012a

Mosek implemenets Matlab bindings for each version. Only include the path for your Matlab version. Type "mosekopt" in Matlab to try out if the path setup is correct.

Installation of Yalmip

Much simpler. Download yalmip, decompress it, and put the folder in your preferred place. Add the path of your yalmip folder "with subfolders" within Matlab. Run "yalmiptest" in Matlab.

The first screen of yalmiptest shows whichever solvers it finds in your system. Mosek should be found if your mosek is installed properly. The second screen summarizes whether each test problem is solved properly or not. If everything looks reasonable (feasible problems solved, infeasible problems unsolved), then you have done yalmip installation.

SOS by Yalmip with Mosek

After setting up your SOS problem, feed the option with the solver set to be "mosek". This can be done by

option = sdpsettings('solver','mosek');

and use this "ops" with the command "solvesos"

solution_info = solvesos(cons, obj, option, var);

The argument "cons" is for constraints, "obj" for the objective function, "var" for decision variables. You will likely notice yalmip prints two errors when you call "sdpsettings"

ERROR - (linprog): Invalid input argument; problem must be a structure.ERROR - (bintprog): Invalid input argument; problem must be a structure.

This is because mosek overloads Matlab internal linprog, bintprog, and quadprog. I don't know if mosek's solver is faster than Matlab's counterparts for linear programming, binary integer programming, and quadratic programming, but Mosek solver does take different options from Matlab's individual solvers. Therefore, Mosek's overloadings are not quite successful in terms of compatibility. Matlab will still use its own "optimset" to set up options and the expected field names and field values may be different from what Mosek expects. Therefore, when "sdpsettings" attempt to collect all options for each solver it supports by calling "optimset(solvername)", for instance, "optimset('linprog')", "optimset" will try to call "solvername('defaults')" to get option structures. However Matlab doesn't document such "defaults" argument in its documentation or anything other than the code itself. Apparently Mosek is not aware of this hidden version of 'defaults' argument, either.

Therefore, Mosek's outputs for "linprog('defaults'), quadprog('defaults'), and bintprog('defaults')" are not compatible with "optimset". To be more precise, Mosek's linprog and bintprog do not allow such a string argument. This is the reason why we get two errors when calling "sdpsettings" (quadprog handles "defaults" so no error printed but the format is still not compatible).

To avoid such ugly error messages, we can suppress them. Because internally Mosek works differently from the simplex method used in "linprog", it may not make sense if we force Mosek to output an option structure that is compatible with Matlab's "optimset" but not compatible with itself. The best bet is to bypass such not informative errors since "solvesos" does not really use the options for linprog, quadprog, and bintprog (need to dig in yalmip code to double check). We can add several lines at the very front of linprog, bintprog, and quadprog:

if ( nargin == 1 && nargout <= 1 && isequal(f,'defaults') )

    return

end

(Change to 'isequal(H, ...' for quadprog)

This will make 'optimset' throw an error that will be caught within 'sdpsettings' instead of invoking Mosek's internal mechanism to handle errors.

Quality compared to Sedumi

SOS is a kind of problem that usually results in slow convergence in optimization. Sedumi in this case reports "run into numerical errors" while Mosek reports "Mosek error: MSK_RES_TRM_STALL ()". Both are referred to the same slow convergence issues. The following statement is quoted from yalmip website: "However, although sum-of-squares converts the original, theoretically intractable, polynomial problem to an SDP which can be solved using convex optimization, the problems that are generated are often huge, and numerically ill-conditioned." As a result, it is suggested in the older version of yalmip website that "The quality of the SOS approximation is typically improved substantially if the tolerance and precision options of the semidefinite solver is decreased. As an example, having sedumi.eps less than 10^-10 when solving sum of squares problems is typically recommended for anything but trivial problems. There is a higher likelihood that the semidefinite solver will complain about numerical problems in the end-phase, but the resulting solutions are typically much better. This seem to be even more important in parameterized problems." The purpose of "sedumi.eps" is explained in Sedumi user guide as that "Sedumi terminates successfully if it finds a solution that violates feasibility and optimality requirements by no more than eps." This is the most detailed statement of the purpose of eps. Other documentation only refers to eps as "desired accuracy," which does not explain clearly enough. Another parameter, "numtol", is suggested not to tamper with them in Addendum to Sedumi User Guide.

Here I list a couple of Sedumi documents as they are usually not easily to be found altogether:

Sedumi user Guide V1.02

Addendum to Sedumi User Guide V1.1

Other info

A poster of Sedumi (intro to other tools seems out-of-date)

An example tutorial

The solution quality of Mosek for SOS problems can be controlled by using the following three options (Section 7.6.5 in Mosek Matlab toolbox manual):

mosek.MSK_DPAR_INTPNT_CO_TOL_PFEAS

mosek.MSK_DPAR_INTPNT_CO_TOL_DFEAS

mosek.MSK_DPAR_INTPNT_CO_TOL_REL_GAP

These three options control the stop criteria of iterations for conic programming. SOS optimization, or essentially SDP, is stated as "a further generalization of conic quadratic optimization." (Section 7.7 in Mosek Matlab toolbox manual) The quality of SOS can be also controlled by these three options.

Besides, the solver iteration log is explained in Section 10.2.2.3 in Mosek Matlab toolbox manual.

Optimization Solver Benchmarks

These days I use a lot of optimization in my research. The optimization method called Sum-of-Squares of polynomials (SOS) programming is the essential backbone of my research topic. Two years ago I started to play with Prof. Megretski's spot (Systems Polynomial Optimization Tools), which is in Matlab and based on Sturm's Sedumi (and sadly just notice Dr. Sturm has passed away since 2003). This spot/Sedumi combination worked fine most of the time, but for more print-out information and options, I followed Prof. Parrilo's suggestion for Yalmip, which has better up-to-date support for SOS. (Interestingly SOS and its first tool SOSTOOLS was created by Parrilo and I was suggested away from his software.) Since then, I have been using Yalmip/Sedumi for most of my work for a while.

However, there are still hiccups for this combination. Sedumi sometimes reports possible bad quality solutions or stops iterations. Yalmip may kindly warn me to check, for example, whether the SOS decomposition is fine. However, that seems not a permanent solution. Therefore, I start to look around for other solvers. Most of optimization problem formulators by default at least support Sedumi and SDPT3. Luckily, Yalmip supports various types of solvers as its own backends. That makes my porting work between solvers much simpler.

So here comes the problem: how to choose between solvers? People argues several criteria, such as solution quality, solution feasibility, speed for certain problems, and more. This note gives some ideas.

And for speed, here is a well-compiled benchmark list, by Prof. Mittelmann at Arizona State University.

Emacs for LaTeX on Windows and on Ubuntu

Want to use Emacs to type up your LaTeX document? Here is the article for you. Setting up Emacs on Ubuntu is a lot easier than on Windows. For Ubuntu (at least on 10.04), simply type

• sudo apt-get install emacs

You may also like to install some other emacs-lisp scripts to help your editing work, for instance, color-theme to give you a better mood while editing and Aspell to enable the on-the-fly spell check in flyspell-mode:

• sudo apt-get install emacs-goodies-el

Although the emacs in the Ubuntu repository is bundled with a simple TeX mode, you may want to use an advanced version: AUCTeX:

• sudo apt-get install auctex

That's all for Ubuntu. Of course here we have assumed that you have had latex installed on your Ubuntu computer.

For Windows, things are trickier. There is no such an automatic installation repository. You have to put things together piece by piece. First of all, you have to find the compatible Emacs version with AUCTeX on Windows. The current supported Emacs version is 23.3. You can find version 24.1 on line, but it is better to install 23.3 just because we prefer AUCTeX for our LaTeX editing work.
Here is the link for Emacs 23.3:

• http://www.gnu.org/software/emacs/

There is actually no installation procedure for Emacs: just download it, decompress it, put the decompressed folder to your preferred location, and make a shortcut so that you can easily launch it. Fr a better mood for editing, you can find color themes here:

• http://www.nongnu.org/color-theme/

Just decompress the file you download and put the folder into your preferred location. In order to let Emacs find the color themes, we have to add the folder into the load list. You can do that by modifying the customized Emacs setup file: .emacs, which is located in ~/AppData/Roaming/ for Windows 7. Note that the file .emacs should be in parallel with the folder emacs.d. If there is no such a file, you should create it yourself. Add the following line in the .emacs file to help Emacs find the folder:

• (setq load-path (append load-path (list "../../el/color-theme-6.6.0")))

We use the relative path here. The current entry of Emacs on load should be the folder bin where Emacs is launched. We create a folder el to place all installed emacs-lisp files in parallel with the folder emacs-23.3. You can choose your preferred location and modify the path being added accordingly. Then you need to link the command so that you can call color-theme by M-x:

• (require 'color-theme)

To link all commands to all themes (a command in color-theme):

• (color-theme-initialize)

And set up the default color theme:

• (color-theme-robin-hood)

which is one of my favor. The theme color-theme-charcoal-black is also pretty good. Restart Emacs in order to see the effect. If the selected theme does not appear, check here for a detailed setup procedure:

• http://www.emacswiki.org/emacs/ColorTheme

Next, we are going to install Aspell and the dictionaries. Aspell is the spell-checking program which is newer than ispell. Anyway, to simply put it, if you need auto-spell checking capability when you are typing in Emacs, you need to install it. You can download it here

• http://aspell.net/win32/

You need to install the program before installing each dictionary. The Aspell version 0.5.3 will not add itself to the path. You can try this command

• aspell -a

to see if the Aspell command is in the path. If not, you have to add it to the path. Once it is in the path, you can use M-x flyspell-mode to check spelling every time when you type a word.

Last, you will need to install AUCTeX. You can find it here:

• http://www.gnu.org/s/auctex/

Installing AUCTeX is relatively easy: just decompress the .zip file, and copy&paste the decompressed folders to overwrite your original Emacs software. Here you have to make sure the AUCTeX version is compatible with the Emacs being overwritten. That's all for Windows!

Enjoy LaTeXing in Emacs!

Embed Fonts in PDF

Got complaints from your publisher about missing fonts? Here we compile a set of commands that you can embed and check whether your file has included everything.

The source is here
http://www.grassbook.org/neteler/highres_pdf.html

http://pages.cs.wisc.edu/~ghost/doc/AFPL/6.50/Ps2pdf.htm

• ps2pdf -dEmbedAllFonts=true main.ps main2.pdf

If you don't have the .ps file with you, try to convert the .pdf file to .ps first

• pdf2ps main.pdf

You can check whether every font is included by

• pdffonts main.pdf

That's all!
Please consult with the 2nd link above for the limitation of ps2pdf. In my case, ps2pdf generates high-resolution bitmapped fonts rather than the original outlined fonts. The bitmapped fonts make the resulting .pdf file pretty ugly on the screen though the print-out is fine.

Any solution for that?

Run MPI Programs in Eclipse with 1-Click

1. Click on the project name in Project Explorer.
2. Right click and choose "Properties"
3. Choose "Run/Debug Settings"
4. Edit one of the launch configurations or create a new one.
5. Under the tab "Main", change C/C++ Application to be "/usr/local/bin/mpirun"
6. Go to the tab "Arguments", change the program arguments to be "-np 4 ${workspace_loc:project_name}/Debug/project_name"

Then you can press "Ctrl+F11" to run your MPI program in Eclipse.

Open MPI Installation

In Ubuntu, key in the following command:

• apt-get install openmpi-bin openmpi-doc libopenmpi-dev
• reference: http://www.cs.ucsb.edu/~hnielsen/cs140/openmpi-install.html 

 In order to test if your setup is correct, download this sample open MPI file

• http://www.eclipse.org/ptp/documentation/2.1/org.eclipse.ptp.pldt.help/html/samples/testMPI.c

Then execute the following commands:

• mpicc -o testMPI testMPI.c
• mpirun -np 4 testMPI 
• reference: http://wiki.eclipse.org/PTP/release_notes/4.0

where the parameter -np specifies the number of cores you would like to run the test file.

How to Use OpenGL with Qt4 in Ubuntu 10.04

When opening an example about OpenGL in Qt Creator, compiling it and running it, you may encounter several errors regarding missing files, such as gl.h and glu.h. They are OpenGL headers. To install the corresponding libraries and headers in Ubuntu 10.04, type the following commands:

• sudo apt-get install libgl1-mesa-dev 
• sudo apt-get install libglu1-mesa-dev

The information about Mesa can be found here:

• http://www.mesa3d.org/

Although the above two commands only request for two libraries, the whole mesa system will be installed as well.

Mesa3D is one of the implementation of OpenGL API. For others, see

• http://en.wikipedia.org/wiki/OpenGL

How to Use Fortran Code in C++

Declare the Fortran function first:

extern "C"{
    // double precision dot product:
    double ddot_(
            const int*        n,      // dimension
            const double*     dx,     // []vector x
            const int*        incx,   // index increment of each access of x
            const double*     dy,     // []vector y
            const int*        incy    // index increment of each access of y
    );
}

Then use it.

How to Publish Source Code in Blogger

There are several ways to do the job. But probably the simplest one is described here. Many ways involve having another host server. The method suggested in the link is based on "Syntaxhighligher." Simply tweak the template HTML file as the following instructions:

1. Go to Layout -> Edit HTML 
   
2. Add the following lines right before </head>
   
   <link href='http://syntaxhighlighter.googlecode.com/svn/trunk/Styles/SyntaxHighlighter.css' rel='stylesheet' type='text/css'/> <script language='javascript' src='http://syntaxhighlighter.googlecode.com/svn/trunk/Scripts/shCore.js'/> <script language='javascript' src='http://syntaxhighlighter.googlecode.com/svn/trunk/Scripts/shBrushCpp.js'/>
    
3. Add the following lines right before </body>
   <script language="javascript">
   dp.SyntaxHighlighter.BloggerMode();
   dp.SyntaxHighlighter.HighlightAll('code');
   </script>
   
   
4. Save the template. Then you are done.

How to publish source code?

1. Type your article as usual.
2. Copy your code to here.
3. Copy your encoded code from the link to where you edit your article.
4. Enclose the code by
   <pre name="code" class="cpp">
   </pre>
   
   
5. Then publish your post as usual.

Install and Link GotoBlas in Eclipse CDT

What is BLAS?
BLAS is the acronym of "Basic Linear Algebra Subprograms." It computes Level 1 (vector-vector), Level 2 (matrix-vector) and Level 3 (matrix-matrix) BLAS operations which are the basis of higher level operations such as linear system solve, least-squared solve, eigen-value decomposition and QR decomposition which are computed in Lapack. The debut of BLAS dates back to 1979 [Lawson1979] written in Fortran and later many higher level packages such as Eispack, Linpack (around 197x-198x) and Lapack (since 1992) built on top of it. Lapack (in Fortran as well) gradually becomes a success and widely used in many programs although many competitors showed up later aiming at its infamous interface (mostly due to the deficiency of Fortran), but Lapack's position in numerical computation seems never shaken by its competitors. Since BLAS is the core of Lapack, BLAS becomes an inevitably important component. (Note 1) Now the term "BLAS" is more like a concept than a specific interface or an implementation: any library which provides Level 1 to Level 3 BLAS operations can be called as a BLAS. For example, Boost::uBlas is a pure C++ implementation of Level 1 to Level 3 BLAS not with a different interface. For backward compatibility, only the one with an "exact" BLAS interface, or a "BLAS implementation" is compatible with Lapack hence interesting. Others implemented with "BLAS functions"  (e.g. uBlas) rather than a "BLAS interface" may grow up as an inevitable library only when an important higher level library relies on it, just as the relationship between Lapack and Blas. (A new and interesting Lapack competitor is Flame, which is also based on BLAS.)

What is GotoBlas?
GotoBlas2 is a Fortran and assembly code implementation of Blas (interface). It is considered the fastest (or one of the few fastest) Blas libraries on the earth. The original developer is Kazushige Goto, who was a research associate in University of Texas at Austin [1]. GotoBlas is manually optimized by writing architecture specific assembly code. Its performance is usually compared with another very fast Blas library: ATLAS. Contrast to the manual optimization strategy of GotoBlas, ATLAS automatically executes several subroutines during its installation on a single machine in order to find the best parameters (e.g. cache sizes). Some experiments report that GotoBlas outperforms ATLAS, for example in [2] and [3].

Why GotoBlas?

1. GotoBlas is very fast as mentioned before. If speed is your first priority
2. It supports multi-threads.  
3. It is available in two licenses: commercial license and open source license. Re-distribution is not allowed but you can always download it here.
4. Lapack is a widely-acceptable dependency library that has been a "must" for many machines for the numerical computation purpose. Even when your software users don't prefer installing a new library, they can still evaluate your code by using the original Lapack and Blas libraries on their machines.
5. Some Lapack functions are built along with GotoBlas, such as some driver routines: LU linear system solve (xGESV), the inverse of a positive definite matrix (xPOTRI),  some computational routines: LU decomposition (xGETRF) and Cholesky factorization (xPOTRF), ans some auxiliary routines. If all you need are to solve a linear system and to do some BLAS operations, then installing GotoBlas suffices.
6. The installation is very easy.
   

How to install GotoBlas?
Very easy. Simply download it, decompress it, make it and move it to the preferred folder. The detailed steps are in the following:

1. Here I use GotoBLAS2-1.13 as an example.
2. Install a Fortran compiler if there is none in your computer (if it is in ubuntu)
   >> sudo apt-get install gfortran
3. Download it here.
4. >> tar -xvf GotoBLAS2-1.13.tar.gz
5. >> cd GotoBLAS2
6. Edit Makefile.rule. Follow the comment instructions.
7. >> make
8. The command "make" will compile the source code according to the rule you wrote in Makefile.rule. It will generate four files in the same folder:
   libgoto2.a
   libgoto2_architecture.13.a
   libgoto2.so
   libgoto2_architecture.13.so
   The first one is the symbolic link to the second one because a general compiler usually searches for a name like it instead of the second one. The second one is the static library itself. architecture changes according to your computer. The third and the last one are the dynamic counterpart of the first two. See here for the knowledge of .a and .so libraries.
   
9. If all you need is the static library. Simply move libgoto2_architecture.13.a to /usr/lib and create a symbolic link to that:
   >> sudo ln -s libgoto2_architecture.13.a libgoto2.a
   Then you are done.
   
10. If you need the dynamic link library, then you need some extra steps. The older version of GotoBlas doesn't compile a dynamic link library by default. But even if this 1.13 version provides .so library, it seems not working properly. Instead, we can still build a dynamic library ourselves:
    >> mkdir temp
    >> cp libgoto2_architecture-r1.13.a temp
    >> cd temp
    >> ar x libgoto2_architecture-r1.13.a
    >> gfortran -shared -lpthread -Wl,-soname,libgoto2.so -o libgoto2.so.1.13 *.o
    
    The meaning of the last command:
1. gfortran: use gfortran to compile
2. -shared: create a shared object
3. -lpthread: use the pthread library. No harm to keep it there if your GotoBLAS is single threaded.
4. -Wl: the options separated by comma after -Wl will be passed to the linker, ld in gcc.
5. -soname,libgoto2.so: passed to ld. It specifies the internal name libgoto2.so of the library which will be used and linked by other programs accordingly. See here for more ideas about ld.
6. -o libgoto2.so.1.13: the file name after compilation
7. *.o: all .o files will be compiled
   
11. Move libgoto2.so.1.13 to the folder you prefer, usually /usr/lib, and create a symbolic link to that
    >> sudo ln -s libgoto2.so.1.13 libgoto2.so 
12. And tell the linker ld that you add a new dynamic library by entering
    >> sudo ldconfig
    So the linker can find it when a program requests to link the new library. Then you are done.

How to link GotoBlas in gcc / gfortran / g++?
Simple. Just add a library option when you compile. For example in g++,
>> g++ -lgoto2 -o yourprogram yourcode.cpp
Notice that -lgoto2 will be interpreted as linking a library called libgoto2.a or libgoto2.so, so don't put both of them in the library search path (-L) or in /usr/lib altogether. That'll be confusing.

How to link GotoBlas in Eclipse
Let's go back to the title. That will be very easy again.

1. Create a HelloWorld project.
2. In Project Explorer, right click on your project name and choose Properties
3. Go to C/C++ Build -> Settings
4. In Tool Settings tab, go to GCC C++ Linker -> Libraries 
5. In Libraries (-l), add "goto2"
6. In Library search path (-L), add the path of libgoto2 if it is not in /usr/lib
7. Then you are done!

Note 1:
We are never able to say a library is the best. A library lasts for many reasons: performance, interface, productivity, available licenses, dependencies, backward compatibility, etc, especially backward compatibility. A new library must provide other way better features such as way faster computation or way more efficient interfaces in order to persuade a software engineer to adopt at the risk of lower publicity because the new library may not be available or has some compatible problems on some machines, or the license may limit the use of the library, so as the software. A famous example of the license issue is about Qt. Before the Qt development company Trolltech was purchased by Nokia, the Qt library only provides GPL and proprietary licenses. The former says the source code should be open as the library you use is. The latter requires very high licensing fee. These two options limit the acceptance of Qt. Now Qt has a new option: LGPL, which says the use of the library is free if it is dynamically linked and the library can be distributed along with the software.  Lapack doesn't have this issue and is free available here.

Ubuntu 10.04 amd64 Desktop on RAID1 Installation

The following are categorized into three parts: general utilities, numerical research related and server utilities.

Install Ubuntu 10.04 on RAID 1

Just don't believe all the above (I put them in the end now) attempts fail. Later I realized I was not alone. Tons of bug reports were posted on-line, e.g. 1, 2, 3, 4, 5, 6, 7, 8. There are at least three different kinds of issues. Most likely there are some bugs in grub2 that cannot handle a RAID more than 2TB and erroneously attempts to access non-existent floating disks even though all simulated ones have been turned off in the bios. Hence a reasonable guess is that this issue is bios dependent (or motherboard dependent). The new version of grub2 is not totally compatible for different One way to workaround is to upgrade from 8.04 LTS to 10.04 LTS. Several has confirmed that would work (see here). This upgrade workaround also works on my Dell Precision T3500 64-bit machine.

• Install Ubuntu 8.04 LTS on RAID1 through Ubuntu 8.04 LTS amd64 Alternate CD. The RAID disk can be created when you create two equal sized partitions in your system. A new item "Create software RAID" will show up.

• Follow the instructions here.
  
  • update-manager --proposed
  • Install updates first.
  • Click "Upgrade" and follow the instructions.

General utilities:

• Archivers
  
  
  • rar: sudo apt-get install rar
  • 7zip: sudo apt-get install p7zip-full
  • both automatically are associated with file-roller (the Gnome default archiver)
    
    
• Multimedia
  
  
  • Go to System -> Administration -> Synaptic
  • Search for ubuntu-restricted-extra and install it
    
    
• Latex
  
  
  • sudo apt-get install texlive-full
  • sudo apt-get install kile
    
    
• Chinese input method
  
  
  • sudo apt-get install gcin
  • set up default input method manager: im-switch -s gcin
  • download the mapping table
    wget http://edt1023.sayya.org/gcin/noseeing-12.tar.gz
  • move the table to the gcin folder
    mv noseeing.gtab ~/.gcin
  • Logout and login
    
    
• Instant messenger
  
  
  • sudo apt-get install kmess
  • enable history and turn off annoying pop-ups and sounds
    
    
• Compiz theme and Emerald decoration
  
  
  • If following the above ubuntu 10.04 installation procedures, Compiz fusion theme utilities are mostly installed. Here are several missing pieces.
  • In System -> Administration -> Synaptic Package Manager,
    search for "Emerald" and install the only entry with the "decorative" keyword in its description.
  • Also in Synaptic, search for "compiz," find the "Compiz Fusion Icon" and install it.
    
  • sudo apt-get install compizconfig-settings-manager
  • Goto System -> Preference -> Startup Applications, add a new entry: fusion-icon --no-start
    
    
• Skype
• Dropbox
• Visualbox
• Tomboy
• nxclient
  

Numerical research related:

• MATLAB
  
  
  • Go to File -> Preferences -> Keyboard to change to Windows key binding style
  • Since mex in Matlab r2009a (both 32-bit and 64-bit versions) only looks for stdc++.so as a standard C++ library, we need to manually make a symbolic link to the ubuntu built-in library.
    cd /usr/lib
    sudo ln -s libstdc++.so.6 libstdc++.so
    
    
• Sedumi 1.3
  
  
  • Add the extracted folder path into Matlab
  • Install Yalmip and run yalmiptest in Matlab. If there is no obvious errors, it's done. Otherwise run "install_sedumi" in Matlab.
    (Note: sometimes the precompiled version along with the sedumi download works fine. But sometimes only the locally compiled version by "install_sedumi" works. It seems always either one fails on a single machine. If you encounter Matlab crashes due to calling the precomiled sedumi, try install_sedumi, and vice versa.)
  
  
• Yalmip R20100512
  
  
  • Extract the downloaded file
  • Add the paths listed in README.txt in Matlab
  • Run "yalmiptest" in Matlab 
   
• spot 250210
  
  
  • Extract the file into the folder "spot" (It will complain later if it is not.)
  • Run "spot_install"
  • Run "spot_check" If the first three pass, it is done. It is because Sedumi cannot provide an optimal solution in the test cases. We only check paths and our setup functionality rather than accuracy here.
  
  
• For others, see here.

Server utilities:

• DynDNS update client (ddclient)
  
  
  • See this very decent step-by-step guide
  • Or you can try this.
    
    
• OpenSSH
  
  
  • See here
  • sudo apt-get install openssh-client
  • sudo apt-get isntall openssh-server
  • Edit /etc/ssh/sshd_config if necessary
    
    
• FTP (SSH can do the job if not using public FTP)
  
  
  • See here
  • 
    
• nxserver
  
  
  • Either way works (can be done before installing SSH server)
    
    
    • http://www.nomachine.com/download.php
    • https://help.ubuntu.com/community/FreeNX
     
    

Miscellaneous:

• Mount partitions at startup
  
  
  • See this
  • The natural and robust way is to modify /etc/fstab directly. For example, one line of my setup is 
  • LABEL=Spring    /media/Spring    ntfs-3g    rw,auto,nls=utf8,uid=1000,gid=1000,umask=0077,fmask=0077     0    0
    
    
    • LABEL=Spring: the name of the partition
    • /media/Spring: mount point
    • ntfs-3g: ntfs format with write ability
    • rw,auto,nls=utf8,uid=1000,gid=1000,umask=0077 (options)
      
      
      • rw: support read and write
      • auto: mount at startup
      • nls=utf8: support uft-8 in file and folder names
      • uid: user id, type "id" in the command line
      • gid: group id
      • umask: file and directory accessibility. It is computed in this way.
      • fmask and dmask are separate setups as umask.
        
    • 0: dump
    • 0: pass. check when reboot if 0.

Appendix:

Not working installation procedure on RAID1

• See here.
  The following are excerpted from the instructions given above.
  
• Use Ubuntu 10.04 to boot your system. Choose "try Ubuntu 10.04"
• Go to System -> Administration -> GParted to make out two equal sized partitions (if you will use two partitions to create a RAID disk.) Make sure reserve some space for swap area.
• sudo apt-get update
• sudo apt-get install mdadm -> Choose "Local Only" when it prompts mail server installation.
• sudo mdadm --create /dev/md1 --verbose --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1
• "Continue creating array?" answer: yes
• Format raid disk: sudo mkfs.ext4 /dev/md1
• Double click on the "Install Ubuntu 10.04 LTS" icon on the desktop.
• Install regularly except choosing “Specify partitions manually (advanced)."
• Specify root/ in md1. Then go through regular installation process.
• Once the installation is done, DON'T REBOOT! Choose "Continue trying Ubuntu 10.04."
• Now we are trying to install mdadm for our new installation. The following instructions just change the root from the installation CD to our new installation spot md1
  
  
  • Mount md1: mount /dev/md1 /target/ (you can also do it by going to "Places")
  • Bind the folders which will be altered by installing mdadm to this installation CD boot
    
    
    • mount --bind /dev/ /target/dev/
    • mount --bind /sys/ /target/sys/
    • mount --bind /proc/ /target/proc/
      
  • Change root to the new installation: chroot /target
• sudo apt-get update
• sudo apt-get install mdadm
  
  
  • Here since I don't need to use Postfix as my mail server. I choose Local again. Don't know whether it affects if I need a mail server later.
    
• Last, install grub to both raid disks in case that when one of them is dead we can still boot from the other.
  
  
  • grub-install /dev/sda
  • grub-install /dev/sdb
• 
  

Install Kubuntu 10.04 directly instead of ubuntu + KDE desktop

I did the latter case first, but I encountered several following problems:

1. gcin input method floating window went dead when the desktop effects are enabled. The dead window made me unable to see which character to choose and it hid a random area on the screen. (It was proved later that this is an issue of gcin. gcin by now cannot function normally in the plasma theme. Installing Kubuntu directly cannot solve this issue)
   
2. Sometimes the background became back to Gnome background and there is no way to change it back rather than rebooting.
3. It is obviously slower than Kubuntu itself.
4. When the desktop effects are enabled, several kinds of windows cannot be displayed when using alt+tab to switch windows.
   

Hence it is not suggested to use both window managers on the same machine.

Here are some tweaks I did after installation kubuntu 10.04:

1. Disable Intel CPU c-state 3 and 4 to stop hissing. (see Ubuntu 9.04, 9.10, 10.04 Installation tweaks in Misc.)
2. Configure power manager of Thinkpad (see Ubuntu 9.04, 9.10, 10.04 Installation tweaks in Misc.)
   
3. Disable annoying kwallet
4. Configure terminal
5. Install KMess and disable online-offline notification
6. Enable desktop effects
7. Right click on the "K" start menu and launch menu editor to add Matlab after installing Matlab in command lines.
8. and many others...

After using Kubuntu 10.04 for a while, there are a couple of issues that I cannot solve so now I go back for Ubuntu 10.04...

1. Mic doesn't work. 
2. Wireless doesn't work. It seemed connected but only few packets are transmitted. In other words, it was not working properly.

Writing: Keywords

This article shows a collection of terms that are usually erroneously used (by myself).

computation time instead of computational time
a large percentage of
a 5% error
3-D instead of 3D

Ubuntu 9.04, 9.10, 10.04 Installation Tweaks

MATLAB

• JRE 
• sudo apt-get install sun-java6-jre
  
  
• VPN 
• sudo aptitude install network-manager-vpnc 
• Panel --> VPN connections --> Configure VPN --> Import the .pcf file
• IPv4 Tab --> click Routes --> uncheck "Use this connection only..."
• click OK, Apply, Close
• Reboot
• Panel --> VPN connections --> MITnet VPN Connection
• MATLAB
• tar -xvf thefile 
• cd thefolder
• sudo ./install...sh

Eclipse

• download: http://www.eclipse.org/downloads/download.php?file=/technology/epp/downloads/release/galileo/SR1/eclipse-cpp-galileo-SR1-linux-gtk.tar.gz
• tar -xvf thefile

Microsoft Office

• WineHQ 
• wget -q http://wine.budgetdedicated.com/apt/387EE263.gpg -O- | sudo apt-key add -
• sudo wget http://wine.budgetdedicated.com/apt/sources.list.d/jaunty.list -O /etc/apt/sources.list.d/winehq.list
• sudo apt-get update
• sudo apt-get install wine
• (some steps may be redundant. need further checks.)
  
  
• WineHQ (for 10.04)
• Just go to Ubuntu Software Center. Search for it and install it.
  
  

• Windows Programs
• Right click on "setup.exe --> Open with "Wine Windows Program Loader"
  

Latex

• TexLive 
• sudo apt-get install texlive (~200 MB) or
• sudo apt-get install texlive-full (~700 MB)
  
  
• TexMaker
  An Latex IDE with fewer KDE library dependencies
  Pros: smaller (~200 MB), looks nice, intuitive configuration
  Cons: not very good auto completion, no block indent 
• sudo apt-get install texmaker
  
• Kile
  An Latex IDE with huge KDE library dependencies
  Pros: looks nicer (much better than the screenshots in its official website) and many utilities
  Cons: fat (~500 MB)
• sudo apt-get install kile
  

Input Method (also works for 9.10 and 10.04)

• noseeing 
• sudo apt-get install gcin 
• set up default input method manager: 
• im-switch -s gcin 
• download the mapping table 
• wget http://edt1023.sayya.org/gcin/noseeing-12.tar.gz 
• move the table to the gcin folder 
• mv noseeing.gtab ~/.gcin  
• Reboot and right click gcin-setup

Numerical Libraries

• Lapack (share)
• sudo apt-get install liblapack3gf 
• sudo ln -s /usr/lib/liblapack.so.3gf /usr/lib/liblapack.so
  
  
• Blas (share)
• sudo apt-get install libblas3gf 
• sudo ln -s /usr/lib/libblas.so.3gf /usr/lib/libblas.so
  
  
• GotoBlas (static)
• download http://www.tacc.utexas.edu/tacc-projects/ 
• tar -xvf thefile 
• cd thefolder 
• ./quickbuild.32bit or 
• gedit Makefile.rule 
• make 
• mkdir temp --> cp libgoto2_penryn-r1.09p3.a ./temp --> cd temp 
• unarchive: ar x libgoto2_penryn-r1.09p3.a 
• recompile into a shared library:
• gfortran -shared -Wl,-soname,libgoto2.so -o libgoto2.so.1.09p3 *.o
• See here for the knowledge of shared libraries. 
• sudo cp libgoto2.so.1.09p3 /usr/lib 
• sudo ldconfig
  

Misc.

• Turn off the terminal beep (9.04 only)
• Terminal --> Profiles --> Edit --> General Tab --> uncheck Terminal bell
  
  
• RAR Archiver 
• sudo apt-get install rar
• sudo ln -fs /usr/bin/rar /usr/bin/unrar

• 7z Archiver 
• sudo apt-get install p7zip-full
  
  
• Skype (works well for 9.04, 9.10, 10.04) 
• http://www.skype.com/intl/zh-Hant/download/skype/linux/choose/ 
• 8.10+ 32-bit works fine on 9.04 32-bit
  
  
• Printer (D-Link DI-707p) 
• System --> Administration --> Printing --> Devices --> Network Printer --> LPD/LPR Host or Printer --> Host: 192.168.0.1 Queue: lp
• Select your printer driver
  
  
• Codec (obsolete)
• sudo apt-get install ubuntu-restricted-extras
  
  
• Battery management (Thinkpad)
• not yet done....
  
  
• CPU hiss "CPU Whine" (from ThinkWiki)
  (for grub2, applicable to ubuntu 9.10 and after)
• sudo joe /etc/default/grub 
• Search for the line GURB_CMDLINE_LINUX="" 
• Replace "" with "processor.max_cstate=2" 
• If the original "" is not empty inside the double quote, simply add processor.max_cstate=2 before the closing quote mark, e.g. replace
• GRUB_CMDLINE_LINUX="acpi_osi="Linux"" with 
• GRUB_CMDLINE_LINUX="acpi_osi="Linux" processor.max_cstate=2"
• sudo update-grub
• sudo reboot
  
• Install restricted codec and more
• Launch Amarok 
• Choose to install all recommended packages
  

Installation List

Windows 7:

7-zip
Anti-vir
Dr. eye
cwtex
Eclipse
GIMP
Google Desktop
Logitech driver
MATLAB
Microsoft Office
Microsoft Visio
FireFox

• AutoPager, Video DownloadHelper, DownThemAll!, Forecastfox, Personal Menu,

Chromifox, Foxdie: neat and configurable

NX Client
PC Man
PDF Creator
PDF Xchange Viewer
RealPlayer
Secure CRT/FX
Skype
WinDjView
Windows Live
X-Win32
紫微論斷

Ubuntu:
Wine
Microsoft Office, Visio
MATLAB
Firefox and its add-ons
Fastcap
GotoBLAS, libblas, liblapack
Eclipse
Nx server/client

Ubuntu server:
OpenSSH
FTP

Scientific Computing / Linear Algebra Library Survey

This is my 101st times surveying linear algebra libraries for C++. Originally I tried to google the performance comparison from one to the other, but unexpectedly this is the most fruitful survey I have ever had since three months ago.

1. Linear Algebra Libraries by Claire Mouton, March 2009.
   Detailed comparison btw CPPLapack, Eigen, Flens, Gmm++, GNU Scientific Library, IT++, Lapack++, MTL, PETSc, Seldon, SparseLib++, TNT, Trilinos, uBlas, and others by listing developers, license, interface, performance description, portability, dependencies, and some limitations. A very clear and useful document.
   
   
2. On the Reusability and Numeric Efficiency of C++ Packages in Scientific Computing
   http://www.linuxclustersinstitute.org/conferences/archive/2003/PDF/Mello_U.pdf
   by Ulisses Mello and Ildar Khabibrakhmanov, IBM T.J. Watson Research Center, Yorktown, NY, USA
   
   Performance comparison by plots in different Blas level 1, 2, 3 experiments. It compares ATLAS, Goto Blas, Blitz, uBlas, STL implementation, Fortran 77 code, C code, bsm, MTL, and A++. It's more like pure performance comparison regardless its interface or whether there is OO wrapper or not. It also provides some observations and reasons why one beats another.
   
   
3. Help choosing a C++ matrix library
   http://old.nabble.com/Help-choosing-a-C%2B%2B-matrix-library-td18857631.html
   In-depth forum discussion. Point out several requirements for a good matrix library.
   

Some webpages regarding specific libraries:

1. Goto Blas
   http://www.utexas.edu/features/2006/goto/
   The author's story. Goto is the author's Japanese last name. GotoBlas is known to be the fastest Blas library in the world. It is handcrafted down to assembly code for different specific architectures.
2. Discussion about the adoption of either uBlas or MTL4
   http://stackoverflow.com/questions/1067821/ublas-vs-matrix-template-library-mtl4
   One mentioned Eigen.
   
3. Eigen benchmark
   http://eigen.tuxfamily.org/index.php?title=Benchmark
   OO abstract, though it provides readable and maintainable code, is always a penalty for performance consideration. Eigen is one of the actively developing libraries which claims its performance. Others as I know are MTL and armadillo. One reason of not claiming performance of their C++ code is because although they provide OO interface, many of them are "wrappers" of Blas and Lapack, like IT++, Lapack++, in order to provide Fortran level performance instead of C++ implementations. It is interesting to note that Eigen claims their code is at a similar speed as ATLAS (Automatically Tuned Linear Algebra Software) and Goto Blas (critically parts are written in assembly). The other thing that is worth noting is that the FLOPS drops when performing vector-vector additions over a certain vector size, while in the IBM report above it happens in matrix-vector multiplication instead. By the way, the performance results are all about Blas level 1, 2, 3. None of them compare solving linear systems or some other advanced decomposition method, like SVD, Cholesky, eigenvalue, etc.
   

Short conclusion: My criteria are OO wraper, namespace, performance, and actively developing or not. I used to be pro MTL4 before I look closer Eigen. Though their is no direct comparison between MTL4 and Eigen (there is a comparison btw MTL2 and Eigen where MTL2 is very slow in Eigen benchmark). MTL4 in their performance report does not beat Goto Blas but the Eigen authors claim they do. Though there is no way to convince myself the comparison is fair, Eigen does provide several "lazy" shortcut and comfortable coding interface compared with MTL4. The other candidate is Boost::uBlas. The advantage of uBlas is it is part of Boost project. That seems to imply more users and better maintenance at the first glance. However, comparing those three, MTL4 seems much more updated than uBlas, and Eigen is more actively updated on its website than the rest two. Eigen also seems more willing to provide other decomposition methods like SVD, eigenvalues.

The last thing: license. LGPL vs. GPL used to be a very big drawback of Qt before version 4.5 since GPL means using the library is equivalent to open your source code to everyone who have access to GPL code (virtually every human being), while LGPL provides flexibility for commercial software and closed code is permitted. Eigen is either GPL or LGPL. It depends on your choice. MTL4 seems not that liberal.

MTL4: MinGW + Eclipse Installation

TDM-MinGW
Eclipse
boost
mtl4

LATEX: Good reference

• cwtex book
• Crash course in Latex
• Arbitrary Latex reference
• The comprehensive Latex symbol
  
• package: listings
  
• package: algorithms
  

NUM: Finite Difference Method readings

Good tutorial, especially for boundary conditions.

Finite Di erence Methods for Di erential Equations
Randall J. LeVeque
UW, 2006

which is first discovered from wikipedia.