Tuesday, July 30, 2013

How to link Intel MKL lapack

This article summarizes how to link existing C++ code with Intel MKL LAPACK without modifying a single line of the original source code that was written with standard LAPACK libraries and compiled by g++.

Intel MKL comprises so many things and for my purpose. In my case, I only want to know if MKL can provide better performance compared to GotoBLAS in my application. So I don't want to change any part of my code and just want to replace GotoBLAS with MKL. This article is written for pure newbies to Intel MKL and is not intended for learning MKL properly.

The task of replacing LAPACK with MKL turns out to be very simple: I used to link my code with GotoBLAS by flag -lgoto2 or standard LAPACK by -llapack. For MKL, this website helps a lot:

http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

After filling my platform information (GNU C/C++ and libgomp), the following flags are suggested:

 -L$MKLROOT/lib/intel64 -lmkl_rt -ldl -lpthread -lm

I replace -libgoto2 with the above flags, recompile my code, and then my code runs smoothly with MKL (and much faster than GotoBLAS for multithreaded cases but slower when single threaded).

This article here summarizes how to control the threads that MKL for the LAPACK part

http://software.intel.com/sites/products/documentation/hpc/mkl/lin/MKL_UG_managing_performance/Using_the_Intel_MKL_Parallelism.htm

In short, MKL automatically uses the number of physical cores as default number of threads. Hyperthreading is not considered as very little juice can be squeezed out when highly optimized and efficient code such MKL is used. If you want to control the number of threads, several global variables can be used and described clearly here:

http://software.intel.com/sites/products/documentation/hpc/mkl/lin/MKL_UG_managing_performance/Using_Additional_Threading_Control.htm

The global variables are independent from OpenMP counterparts. In my case, for example, when I want to use two threads only to solver my linear system Ax=b, I enter the following command in bash:

export MKL_NUM_THREADS=2

before I run my executable. Different performance being observed before and after setting up the global variable confirms the success of my attempt.







Tuesday, June 4, 2013

Yalmip with Mosek in SOS optimization

This post describes how to make Mosek work smoothly with Yalmip in SOS program. The following  guide is for academic use on Linux Mint 15 (Ubuntu 13.04) with Matlab R2012a, Mosek 7.0.0.65, and Yalmip R20130405.

Installation of Mosek

You simply download it from Mosek.com, untar the downloaded file directly under your home folder, apply for the academic license, and place the license file directly under the mosek folder. The mosek folder and license place are rigid. Changing them may make mosek not function properly.

Mosek for Linux is precompiled and only shipped in .a and .so files. So you won't encounter make hiccups and you can't easily decipher what is going on within. You may either have it work like a breeze or not work at all. And there is nothing you can do if in the second case. (Or few things but anyway not much like cracking the code because the source is not shipped.)

If you want to use mosek in command line, edit your .bashrc with this additional line:

export PATH="$PATH:$HOME/mosek/7/tools/platform/linux64x86/bin"

Then "source .bashrc" or restart your terminal. You can try type "mosek" in your new open-up terminal to see if the path has been set up correctly.

If you want to use mosek in Matlab only, set up mosek path in your Matlab search path:

/home/youraccount/mosek/7/toolbox/r2012a

Mosek implemenets Matlab bindings for each version. Only include the path for your Matlab version. Type "mosekopt" in Matlab to try out if the path setup is correct.

Installation of Yalmip

Much simpler. Download yalmip, decompress it, and put the folder in your preferred place. Add the path of your yalmip folder "with subfolders" within Matlab. Run "yalmiptest" in Matlab.

The first screen of yalmiptest shows whichever solvers it finds in your system. Mosek should be found if your mosek is installed properly. The second screen summarizes whether each test problem is solved properly or not. If everything looks reasonable (feasible problems solved, infeasible problems unsolved), then you have done yalmip installation.

SOS by Yalmip with Mosek

After setting up your SOS problem, feed the option with the solver set to be "mosek". This can be done by

option = sdpsettings('solver','mosek');

and use this "ops" with the command "solvesos"

solution_info = solvesos(cons, obj, option, var);

The argument "cons" is for constraints, "obj" for the objective function, "var" for decision variables. You will likely notice yalmip prints two errors when you call "sdpsettings"

ERROR - (linprog): Invalid input argument; problem must be a structure.ERROR - (bintprog): Invalid input argument; problem must be a structure.

This is because mosek overloads Matlab internal linprog, bintprog, and quadprog. I don't know if mosek's solver is faster than Matlab's counterparts for linear programming, binary integer programming, and quadratic programming, but Mosek solver does take different options from Matlab's individual solvers. Therefore, Mosek's overloadings are not quite successful in terms of compatibility. Matlab will still use its own "optimset" to set up options and the expected field names and field values may be different from what Mosek expects. Therefore, when "sdpsettings" attempt to collect all options for each solver it supports by calling "optimset(solvername)", for instance, "optimset('linprog')", "optimset" will try to call "solvername('defaults')" to get option structures. However Matlab doesn't document such "defaults" argument in its documentation or anything other than the code itself. Apparently Mosek is not aware of this hidden version of 'defaults' argument, either.

Therefore, Mosek's outputs for "linprog('defaults'), quadprog('defaults'), and bintprog('defaults')" are not compatible with "optimset". To be more precise, Mosek's linprog and bintprog do not allow such a string argument. This is the reason why we get two errors when calling "sdpsettings" (quadprog handles "defaults" so no error printed but the format is still not compatible).

To avoid such ugly error messages, we can suppress them. Because internally Mosek works differently from the simplex method used in "linprog", it may not make sense if we force Mosek to output an option structure that is compatible with Matlab's "optimset" but not compatible with itself. The best bet is to bypass such not informative errors since "solvesos" does not really use the options for linprog, quadprog, and bintprog (need to dig in yalmip code to double check). We can add several lines at the very front of linprog, bintprog, and quadprog:

if ( nargin == 1 && nargout <= 1 && isequal(f,'defaults') )
    return
end

(Change to 'isequal(H, ...' for quadprog)
This will make 'optimset' throw an error that will be caught within 'sdpsettings' instead of invoking Mosek's internal mechanism to handle errors.

Quality compared to Sedumi

SOS is a kind of problem that usually results in slow convergence in optimization. Sedumi in this case reports "run into numerical errors" while Mosek reports "Mosek error: MSK_RES_TRM_STALL ()". Both are referred to the same slow convergence issues. The following statement is quoted from yalmip website: "However, although sum-of-squares converts the original, theoretically intractable, polynomial problem to an SDP which can be solved using convex optimization, the problems that are generated are often huge, and numerically ill-conditioned." As a result, it is suggested in the older version of yalmip website that "The quality of the SOS approximation is typically improved substantially if the tolerance and precision options of the semidefinite solver is decreased. As an example, having sedumi.eps less than 10-10 when solving sum of squares problems is typically recommended for anything but trivial problems. There is a higher likelihood that the semidefinite solver will complain about numerical problems in the end-phase, but the resulting solutions are typically much better. This seem to be even more important in parameterized problems." The purpose of "sedumi.eps" is explained in Sedumi user guide as that "Sedumi terminates successfully if it finds a solution that violates feasibility and optimality requirements by no more than eps." This is the most detailed statement of the purpose of eps. Other documentation only refers to eps as "desired accuracy," which does not explain clearly enough. Another parameter, "numtol", is suggested not to tamper with them in Addendum to Sedumi User Guide.

Here I list a couple of Sedumi documents as they are usually not easily to be found altogether:

Other info
A poster of Sedumi (intro to other tools seems out-of-date)

The solution quality of Mosek for SOS problems can be controlled by using the following three options (Section 7.6.5 in Mosek Matlab toolbox manual):

mosek.MSK_DPAR_INTPNT_CO_TOL_PFEAS
mosek.MSK_DPAR_INTPNT_CO_TOL_DFEAS
mosek.MSK_DPAR_INTPNT_CO_TOL_REL_GAP

These three options control the stop criteria of iterations for conic programming. SOS optimization, or essentially SDP, is stated as "a further generalization of conic quadratic optimization." (Section 7.7 in Mosek Matlab toolbox manual) The quality of SOS can be also controlled by these three options.

Besides, the solver iteration log is explained in Section 10.2.2.3 in Mosek Matlab toolbox manual.

Monday, June 3, 2013

Optimization Solver Benchmarks

These days I use a lot of optimization in my research. The optimization method called Sum-of-Squares of polynomials (SOS) programming is the essential backbone of my research topic. Two years ago I started to play with Prof. Megretski's spot (Systems Polynomial Optimization Tools), which is in Matlab and based on Sturm's Sedumi (and sadly just notice Dr. Sturm has passed away since 2003). This spot/Sedumi combination worked fine most of the time, but for more print-out information and options, I followed Prof. Parrilo's suggestion for Yalmip, which has better up-to-date support for SOS. (Interestingly SOS and its first tool SOSTOOLS was created by Parrilo and I was suggested away from his software.) Since then, I have been using Yalmip/Sedumi for most of my work for a while.

However, there are still hiccups for this combination. Sedumi sometimes reports possible bad quality solutions or stops iterations. Yalmip may kindly warn me to check, for example, whether the SOS decomposition is fine. However, that seems not a permanent solution. Therefore, I start to look around for other solvers. Most of optimization problem formulators by default at least support Sedumi and SDPT3. Luckily, Yalmip supports various types of solvers as its own backends. That makes my porting work between solvers much simpler.

So here comes the problem: how to choose between solvers? People argues several criteria, such as solution quality, solution feasibility, speed for certain problems, and more. This note gives some ideas.

And for speed, here is a well-compiled benchmark list, by Prof. Mittelmann at Arizona State University.


Saturday, September 17, 2011

Emacs for LaTeX on Windows and on Ubuntu

Want to use Emacs to type up your LaTeX document? Here is the article for you. Setting up Emacs on Ubuntu is a lot easier than on Windows. For Ubuntu (at least on 10.04), simply type
  • sudo apt-get install emacs
You may also like to install some other emacs-lisp scripts to help your editing work, for instance, color-theme to give you a better mood while editing and Aspell to enable the on-the-fly spell check in flyspell-mode:
  • sudo apt-get install emacs-goodies-el
Although the emacs in the Ubuntu repository is bundled with a simple TeX mode, you may want to use an advanced version: AUCTeX:
  • sudo apt-get install auctex
That's all for Ubuntu. Of course here we have assumed that you have had latex installed on your Ubuntu computer.

For Windows, things are trickier. There is no such an automatic installation repository. You have to put things together piece by piece. First of all, you have to find the compatible Emacs version with AUCTeX on Windows. The current supported Emacs version is 23.3. You can find version 24.1 on line, but it is better to install 23.3 just because we prefer AUCTeX for our LaTeX editing work.
Here is the link for Emacs 23.3:
There is actually no installation procedure for Emacs: just download it, decompress it, put the decompressed folder to your preferred location, and make a shortcut so that you can easily launch it. Fr a better mood for editing, you can find color themes here:
Just decompress the file you download and put the folder into your preferred location. In order to let Emacs find the color themes, we have to add the folder into the load list. You can do that by modifying the customized Emacs setup file: .emacs, which is located in ~/AppData/Roaming/ for Windows 7. Note that the file .emacs should be in parallel with the folder emacs.d. If there is no such a file, you should create it yourself. Add the following line in the .emacs file to help Emacs find the folder:
  • (setq load-path (append load-path (list "../../el/color-theme-6.6.0")))
We use the relative path here. The current entry of Emacs on load should be the folder bin where Emacs is launched. We create a folder el to place all installed emacs-lisp files in parallel with the folder emacs-23.3. You can choose your preferred location and modify the path being added accordingly. Then you need to link the command so that you can call color-theme by M-x:
  • (require 'color-theme)
To link all commands to all themes (a command in color-theme):

  • (color-theme-initialize)
And set up the default color theme:
  • (color-theme-robin-hood)
which is one of my favor. The theme color-theme-charcoal-black is also pretty good. Restart Emacs in order to see the effect. If the selected theme does not appear, check here for a detailed setup procedure:

Next, we are going to install Aspell and the dictionaries. Aspell is the spell-checking program which is newer than ispell. Anyway, to simply put it, if you need auto-spell checking capability when you are typing in Emacs, you need to install it. You can download it here
You need to install the program before installing each dictionary. The Aspell version 0.5.3 will not add itself to the path. You can try this command
  • aspell -a
to see if the Aspell command is in the path. If not, you have to add it to the path. Once it is in the path, you can use M-x flyspell-mode to check spelling every time when you type a word.

Last, you will need to install AUCTeX. You can find it here:
Installing AUCTeX is relatively easy: just decompress the .zip file, and copy&paste the decompressed folders to overwrite your original Emacs software. Here you have to make sure the AUCTeX version is compatible with the Emacs being overwritten. That's all for Windows!

Enjoy LaTeXing in Emacs!

Friday, September 16, 2011

Embed Fonts in PDF

Got complaints from your publisher about missing fonts? Here we compile a set of commands that you can embed and check whether your file has included everything.

The source is here
http://www.grassbook.org/neteler/highres_pdf.html
 
  • ps2pdf -dEmbedAllFonts=true main.ps main2.pdf
If you don't have the .ps file with you, try to convert the .pdf file to .ps first
  • pdf2ps main.pdf
You can check whether every font is included by
  • pdffonts main.pdf
That's all!
Please consult with the 2nd link above for the limitation of ps2pdf. In my case, ps2pdf generates high-resolution bitmapped fonts rather than the original outlined fonts. The bitmapped fonts make the resulting .pdf file pretty ugly on the screen though the print-out is fine.

Any solution for that?

Saturday, October 30, 2010

Run MPI Programs in Eclipse with 1-Click

  1. Click on the project name in Project Explorer.
  2. Right click and choose "Properties"
  3. Choose "Run/Debug Settings"
  4. Edit one of the launch configurations or create a new one.
  5. Under the tab "Main", change C/C++ Application to be "/usr/local/bin/mpirun"
  6. Go to the tab "Arguments", change the program arguments to be "-np 4 ${workspace_loc:project_name}/Debug/project_name"
Then you can press "Ctrl+F11" to run your MPI program in Eclipse.

Open MPI Installation

In Ubuntu, key in the following command:

 In order to test if your setup is correct, download this sample open MPI file
Then execute the following commands:
where the parameter -np specifies the number of cores you would like to run the test file.