Monday 4 January 2010

Compile Code_Saturne with SALOME binary libraries

As instructed in "Installation of Code_Saturne 2.0.0 on Ubuntu 9.04", Code_Saturne needs libraries openmpi and metis to support parallel computing, and needs libraries hdf and med to support the MED file format. In the previous post, openmpi was installed by apt-get, however for the other three, metis, hdf and med, the libraries were built from the source code.

If, incidentally, you are also a SALOME user, it is possible to build Code_Saturne with the libraries shipped in the SALOME package. The advantage of doing this is saving time to download the latest source packages and compile them manually. Therefore, briefly, this post introduces the new compilation procedure of CS 2.0.0 with SALOME libraries.

1. install necessary packages using apt-get

:/$ sudo apt-get install build-essential gfortran libxml2 libxml2-dev libatlas-headers libatlas-base-dev openmpi-bin openmpi-dev libibverbs-dev openssh-server python-qt4 pyqt4-dev-tools

2. install SALOME. I assume you install SALOME 5.1.3, the latest version currently

Posts on SALOME installation can be referred and help on this step.

3. compile the four parts, bft, fvm, ecs and ncs. I assume you installed SALOME into "/home/salad/salome_5.1.3/", in which "salad" is my user name, and you have to use yours instead.

# ship into bft-1.1.1
:/$ ./configure
:/$ make
:/$ sudo make install
# ship into fvm-0.14.0
:/$ ./configure --with-hdf5=/home/salad/salome_5.1.3/hdf5-1.6.9 --with-med=/home/salad/salome_5.1.3/med-2.3.6 --with-mpi=/usr/lib/openmpi
:/$ make
:/$ sudo make install
# ship into ecs-2.0.0-beta2
:/$ ./configure --with-hdf5=/home/salad/salome_5.1.3/hdf5-1.6.9 --with-med=/home/salad/salome_5.1.3/med-2.3.6 --with-metis-include=/home/salad/salome_5.1.3/metis-4.0/Lib --with-metis-lib=/home/salad/salome_5.1.3/metis-4.0
:/$ make
:/$ sudo make install
# ship into ncs-2.0.0-beta2
:/$ ./configure --with-mpi=/usr/lib/openmpi --with-prepro=/usr/local LIBS=-lm
:/$ make
:/$ sudo make install

27 comments:

  1. I installed code saturne following this guide. The installation ended without error, but I have a question. How can I run Code Saturne. I tried to launch the cs script (with double clic) or using the terminal pointing /usr/local/bin, buth nothing happen.
    Thanks
    Simone

    ReplyDelete
  2. I planned to write a simple example on how to use Code_Saturne, but you know, life is always busy. Can you have a look at my new post:

    http://code-saturne.blogspot.com/2010/02/cfd-example-laminar-flow-along-2d_06.html

    I hope it helps.

    I am still working on this series of posts and as such the content will be updated everyday. Your comments will also help me make it better and better.

    Regards,

    ReplyDelete
  3. -------------------------
    A. INSTALLATION OF SALOME
    -------------------------

    Salome installed in /opt/Salome_5.1.3.

    (I do not remember if I checked the check button "Install modules to a single directory")

    Runs fine, import *.hdf file created on another installation, switches between Geometry &
    Mesh modules, exports mesh as *.unv & *.med.

    ---------------------------
    B. INSTALLATION OF CFDSTUDY
    ---------------------------

    4.1) inside your salome installation edit the env_products.sh file or the env_products.csh file
    I do not have an env_products.sh file, but I do have a envSalome-V5_1_3.sh, located in the /opt/Salome_5.1.3/Salome5/V5_1_3 directory to which I have added the lines specified.

    #------ CFDSTUDY ------
    export CFDSTUDY_ROOT_DIR=${INST_ROOT}/CFDSTUDY_2.0.0
    if [ -n "${ENV_FOR_LAUNCH}" ] ; then
    if [ "${ENV_FOR_LAUNCH}" == "1" ] ; then
    exportp PATH ${CFDSTUDY_ROOT_DIR}/bin/salome
    exportp PYTHONPATH ${CFDSTUDY_ROOT_DIR}/bin/salome:${CFDSTUDY_ROOT_DIR}/lib/python${PYTHON_VERSION}/site-packages/salome
    fi
    fi
    ##
    #------ CFDSTUDY_src ------
    export CFDSTUDY_SRC_DIR=${INST_ROOT}/CFDSTUDY_SRC

    4.2) inside the KERNEL_5.1.3 directory from your salome installation edit ithe salome.sh or salome.csh file
    and modify it as the same way as 4.1
    I do not have a salome.sh or salome.csh file in the KERNEL_V5_1_3 directory, nor in the KERNEL_V5_1_3/bin/salome directory

    Should I be looking at the "runSalome.sh" files?

    5) inside your salome installation update the environment: source env_products.sh- Done in the /opt/Salome_5.1.3/Salome5/V5_1_3 directory, returns with no error.

    6-7) Unpacked CFDSTUDY-2.0 in the /opt/Salome_5.1.3 directory, downloaded the cfd_study_install.sh file independently, into the same directory.

    8) in the directory CFDSTUDY-2.0 do: chmod +x cfdstudy_install.sh- Done

    9) run the installation script: ./cfdstudy_install.sh- results in:

    Warning: the environment variable INST_ROOT is not set,
    the Salome environment must be set
    do: source env_products.sh
    or: source env_products.csh

    The installation stops.

    Where is INST_ROOT supposed to be set? What should INST_ROOT be Set to?
    Repeating step 5 above has no effect whatsoever.

    My Tree looks like this:

    /
    opt
    Salome_5.1.3
    CFDSTUDY-2.0
    SALOME5
    V5_1_3
    KERNEL_V5_1_3


    Working on Ubuntu 9.10 64 bit...

    Please help...

    ReplyDelete
  4. Anonamous-
    I have had some real heart burn trying to get Code_Saturne 1.3.3 running under Ubuntu 9.10 myself, and possibly some of my suggestions here will be of use to you.

    The first thing I did was add the following environmental variables to my /etc/environment file:

    CS_HOME="/home/rocketman/Saturne/Noyau/ncs-1.3.3"
    ECS_HOME="/home/rocketman/Saturne/Enveloppe/ecs-1.3.3"
    CSGUI_HOME="/home/rocketman/Saturne/Interface/ics-1.3.3"
    SYRCS_HOME="/home/rocketman/Saturne/opt/syr_cs-2.1.1"
    CS_ROOT="/home/rocketman/Saturne"

    This is most likely not the appropriate place to set these variables, but since this is a single-user installation that will be dedicated to this work, I don't have a problem with permanently setting these variables every time I boot.

    Now, in your /Noyau/ncs-1.3.3/bin directory is an executable called "cree_sat", and in your Saturne base directory is the executable "cs_create". These apparently do the same thing that the "SaturneStudy.py" script does under CAELinux, which is create a new study- a small GUI that asks for Study Name, Case Name, Base Directory and mesh file, then creates the study file system. The difference appears to be that "cree_sat" works in the original French, and "cs_create" works in English. But, in my case, the "cs_create" for some reason does not create all of the correct directories in the Study file system, and the python script I imported from CAELinux doesn't like how I set my system up. Once the file structure is properly set up, I still have to import the mesh file to the "Maillage" folder (*.unv instead of *.med in my case, because I still don't have med support set up right). Once you have the Study file system set up, look in the "[STUDY]/[CASE_Number]/DATA directory, and you should find an executable, "SaturneGUI", which starts the actual main GUI where you preprocess the mesh, set the various study parameters and ultimately (if successful) get a full analysis. I have gotten to the point where everything works as it should until the final calculations, at which time it crashes. So, there is still something I am not doing right.

    I am seriously thinking about abandoning this first installation and trying the newer beta version, or the CFDSTUDY package that weds Saturne to Salome...CAELinux was so much easier than trying to build this myself...

    ReplyDelete
  5. In your instruction #1 above, you specify the necessary package "openmpi-dev", which is the only one I am lacking. When I run apt-get for this, I get the following results:

    Package openmpi-dev is a virtual package provided by:
    libopenmpi-dev 1.3.2-3ubuntu1.1
    You should explicitly select one to install.
    E: Package openmpi-dev has no installation candidate

    Since I have libopenmpi-dev 1.3.2-3ubuntu.1 already installed on my system, does this mean I am covered?

    ReplyDelete
  6. Hi Charlie,

    I didn't try CFDSTUDY yet, but I try to answer your questions.

    1. Did you use "Universal binaries for Linux" to install SALOME. It is quite strange because the files you are missing are all found in my directories. I used the install wizard version to install SALOME, which I suggest. Unfortunately, I don't know about the universal binaries version.

    2. For Code_Saturne I suggest the newest version 2.0, even though it is still at a RC stage. Follow my instructions it won't be difficult to compile by yourself, I guess. Leave comments and I will try to help you.

    I use Code_Saturne independent from SALOME.

    3. You are right on openmpi-dev. After you have libopenmpi-dev, openmpi-dev is then okay. They always change the package policy, and therefore I have to double check and update my post again.

    Best regards,

    ReplyDelete
  7. Thanks for the response. I used the Universal binaries for Salome. The actual file was Salome_5.1.3_64bits.tgz. I had tried the Salome-MECA-2009.1-GPL before, which gave me a Version 4.1 something of Salome, but that one would crash the system when trying to shift in to the geometry mode- moved on to the Salome_5.1.3 and that runs fine on a problem I had put together on my CAELinux machine, including Post-Pro on a Code Saturne solution from CAELinux. I do notice my new version of Salome does not bundle Aster as the earlier versions did...I am taking your advice and moving on to the Version 2.0 of Code Saturne- I don't mind using Code Saturne independently of Salome, since that is how I do it now anyway. What I want to be sure of is to incorporate multiprocessor support in the Code Saturne solver, since the single processor takes several hours to get through a solution in some cases- anything to speed that up!
    Do I need to link openmpi-dev to libopenmpi-dev for Code Saturne to find it?

    ReplyDelete
  8. I have made several attempts at building Code_Saturne-2.0.0-beta2, following your instructions, then modifying a few things, then trying the install script- nothing works. I get down to the point of making ncs, and wind up with the following error, no matter what I do:

    libtool: link: gcc -funroll-loops -O2 -Wuninitialized -std=c99 -funsigned-char -pedantic -W -Wall -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -Wunused -Wfloat-equal -rdynamic -O -o .libs/cs_solver ./.libs/libsaturne.so -L/opt/salome_5.1.3/SALOME5/Prerequis/Hdf5-1.6.9/lib -L/usr/lib/openmpi/lib /usr/lib/libfvm.so -lfvm_coupl /usr/lib/libhdf5.so /usr/lib/libbft.so -lz /usr/lib/openmpi/lib/libmpi.so /usr/lib/libibverbs.so -lpthread /usr/lib/openmpi/lib/libopen-rte.so /usr/lib/openmpi/lib/libopen-pal.so -lnsl -lutil /usr/lib/libxml2.so -lcblas -latlas -L/usr/lib/gcc/x86_64-linux-gnu/4.4.1 -L/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib -L/lib/../lib -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../.. -L/usr/lib/x86_64-linux-gnu -lgfortranbegin -lgfortran -lm -ldl -pthread -Wl,-rpath -Wl,/usr/lib/openmpi/lib
    ./.libs/libsaturne.so: undefined reference to `fvm_part_to_block_create_by_gnum'
    ./.libs/libsaturne.so: undefined reference to `fvm_order_local_i'
    ./.libs/libsaturne.so: undefined reference to `fvm_neighborhood_create'
    ./.libs/libsaturne.so: undefined reference to `fvm_parall_get_mpi_comm'
    ./.libs/libsaturne.so: undefined reference to `fvm_block_to_part_destroy'
    ./.libs/libsaturne.so: undefined reference to `fvm_io_num_create_from_adj_i'
    ./.libs/libsaturne.so: undefined reference to `fvm_block_to_part_copy_index'
    ./.libs/libsaturne.so: undefined reference to `fvm_neighborhood_set_options'
    ./.libs/libsaturne.so: undefined reference to `fvm_neighborhood_transfer_data'
    ./.libs/libsaturne.so: undefined reference to `fvm_parall_counter_max'
    ./.libs/libsaturne.so: undefined reference to `fvm_block_to_part_get_n_part_ents'
    ./.libs/libsaturne.so: undefined reference to `fvm_nodal_copy_edges'
    ./.libs/libsaturne.so: undefined reference to `fvm_block_to_part_create_by_adj_s'
    ./.libs/libsaturne.so: undefined reference to `fvm_block_to_part_transfer_gnum'
    ./.libs/libsaturne.so: undefined reference to `fvm_neighborhood_get_box_stats'
    ./.libs/libsaturne.so: undefined reference to `fvm_block_to_part_create_by_gnum'
    ./.libs/libsaturne.so: undefined reference to `fvm_neighborhood_by_boxes'
    ./.libs/libsaturne.so: undefined reference to `fvm_part_to_block_destroy'
    ./.libs/libsaturne.so: undefined reference to `fvm_block_to_part_create_adj'
    ./.libs/libsaturne.so: undefined reference to `fvm_block_to_part_copy_array'
    ./.libs/libsaturne.so: undefined reference to `fvm_neighborhood_destroy'
    ./.libs/libsaturne.so: undefined reference to `fvm_parall_set_mpi_comm'
    ./.libs/libsaturne.so: undefined reference to `fvm_block_to_part_create_by_rank'
    ./.libs/libsaturne.so: undefined reference to `fvm_neighborhood_get_times'
    ./.libs/libsaturne.so: undefined reference to `fvm_block_to_part_copy_indexed'
    ./.libs/libsaturne.so: undefined reference to `fvm_parall_counter'
    ./.libs/libsaturne.so: undefined reference to `fvm_part_to_block_copy_array'
    collect2: ld returned 1 exit status
    make[3]: *** [cs_solver] Error 1
    make[3]: Leaving directory `/opt/Saturne-2.0.0-beta2/ncs-2.0.0-beta2/src/apps'
    make[2]: *** [all-recursive] Error 1
    make[2]: Leaving directory `/opt/Saturne-2.0.0-beta2/ncs-2.0.0-beta2/src'
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/opt/Saturne-2.0.0-beta2/ncs-2.0.0-beta2'
    make: *** [all] Error 2

    So, where do I go from here?

    ReplyDelete
  9. Hi Charlie,

    I suggest to use the install wizard for SALOME, although the package seems bigger. Because I don't use Aster, I tried Salome-MECA-2009.1-GPL long time ago, but I don't use it now. I think an independent SALOME for Pre-Pro & Post-Pro, and a seperated Code_Saturne for solving is okay for me. Install them on a fresh Ubuntu is clear.

    From your questions I feel my posts become not as clear as I expected. I am sorry for this. Software, Ubuntu, SALOME and Code_Saturne, are all developing, and I also have to spare time to summarise them again in new posts. Ok, I can try to help you tackle the problems.

    Firstly, recommend you to read the top post of my blog, "README". Two posts are mentioned for the installation of the latest Code_Saturne 2.0-rc1. I suggest to read them all to get a general idea before you trying to compile and install. A common order to compile is described in

    http://code-saturne.blogspot.com/2009/08/installation-of-codesaturne-200-on.html

    and some changes in 2.0-rc1 (strongly recommended, because of the better MEI support) is said in

    http://code-saturne.blogspot.com/2010/03/installation-of-codesaturne-20-rc1-on.html

    Briefly, the basic idea is

    1. Install necessary packages by apt-get. openmpi is for multi-core calculation. Don't worry. Following the instrctions, Code_Saturne will find it.

    2. Compile metis for multi-core calculation (I have a patched metis for newer Ubuntu, find it in my posts and use it). Compile HDF5 and MED if you use MED file format for Pre-Pro or Post-Pro.

    (If you want to re-use them from SALOME, follow the present post.)

    (David also suggested a way to install HDF5 and MED by apt-get, but I haven't succeeded yet.)

    3. Sequentially compile bft, fvm, mei, ecs and ncs. For each of them, three commands are used to compile:

    $ ./configure [options]
    $ make
    $ sudo make install

    Exactly following my posts to give options for the first command, ./configure, it should be okay.

    The error implies the linker could not find the symbols from fvm. From the compiling line, your fvm and bft libraries are

    /usr/lib/libfvm.so
    /usr/lib/libbft.so

    You should check the location, because by my instructions bft and fvm are supposed to be installed into /usr/local, but you mentioned a location in /usr.

    Good luck and don't hesitate to communicate.

    Best regards,

    ReplyDelete
  10. I find that I have multiple /usr/lib/libfvm,so and /usr/lib/libbft.so files (libfvm.so, libfvm.so.0, libfvm.so.0.0, libfvm1a, etc.)- also, multiple copies of these libs in /usr/local/lib. I suspect all the different attempts to build the package may have resulted in confusing the system. Also, I still have an almost-running version of 1.3.3 installed that may be confusing things. My Salome installation (5.1.3) appears to function properly. I can read *.hdf and *.med files created in CAELinux, and manipulate them at will, and all the packages (med, hdf5, metis) are resident. Everything appears to work as you say it should work, until I try making ncs- no complaints about any of the opptions, everything seems to configure OK. I will try hiding the old libs and try again...

    ReplyDelete
  11. OK, made progress. I started with a clean set-up- threw away all the previous libraries I created. Had to abandon mei for the time being because it would not make, but now it appears I have a functional installation. bft and fvm want to write the *.so files (and several others) into the /usr/local/lib directory for some reason. By making links in /usr/lib to these files, I was able to get it to build. Now to figure out how to make this work. Thanks for all the help- I couldn't have gotten this far without your guidance...

    ReplyDelete
  12. OK. So how do I build the documents for Saturne 2.0.0-beta2? I have downloaded the referenced packages, try to run the Makefile, and get nothing but errors (unexpected ";", files missing, etc.). is there a quick tutorial on how to get the documentation in a readable format?

    ReplyDelete
  13. When the commands "make pdf" "make install-pdf" are done, the documentation is compiled and installed in the PREFIX/share/doc/ncs for the kernel, where PREFIX is the root of your installation. Note that the installation process has been improved between versions 2.0-beta2 and 2.0-rc1 (thanks to the debian packaging).

    ReplyDelete
  14. Hi,

    Thanks for Alexandre's help :)

    Charlie, libfvm.so, libfvm.so.0 and libfvm.so.0.0 etc, actually, only one of them is real and others are symbols linking to the real copy. You can use command 'ls -l' to check it. They don't have the same colour if you 'ls' them on terminal.

    I suggest to install compiled software into /usr/local, because the deb packaged ones will be installed into /usr, commonly.

    If you don't use latex, you can find a link from my post to directly download the pdf manuals I compiled

    http://code-saturne.blogspot.com/2010/03/installation-of-codesaturne-20-rc1-on.html

    Best regards,

    ReplyDelete
  15. Alexandre- thanks on the "make pdf" command- I did not know that one had to add the "pdf" part.
    With regards to the installation, I find that I need instruction on the cs_create command- which works, I just don't know how to set the command line parameters (CAELinux gave a nice GUI for creating new studies, which got my file structure set up where I wanted it). I think GUI's are a tremendous improvement over the old style (I go back to the punch card days, where the mesh was defined by a stack of cards, and a single comma mis-placed meant one had to go through the stack card by card...GUI I can see what my parameters are set to before I hit the run case button).
    I am not happy with my current build. The GUI does not offer the preprocessor where it used to be, and I can't figure out yet how to set boundary conditions if I don't preprocess. I would also like to add mei and blas and probably sythres (although I don't have much use for that yet). Anything to cut the run time down from my current eitght hours or so on a small case!
    What is the status of the *.deb package development? Will it function under 9.10, or will one have to move up to 10.04? Should I wait to rebuild the package until the *.deb is available, or should I try tweaking it now? Seems most of what I want to do can be done by only rebuilding ncs...
    Again, both salad and Alexandre- thanks for your help.

    ReplyDelete
  16. Hi Charlie,

    You are welcome.

    In the pdf files there is a 'refcard.pdf'; please briefly look through it. A command with option '--help' will also give a short prompt on how to use the command.

    I think you can try CFDSTUDY because it provides a combination from Code_Saturne to the platform SALOME. Code_Saturne is only a solver, and therefore when used stand-alone, you have to know well about the mesh model you are using to set boundaries.

    I guess you don't necessarily wait for the deb packages to use blas. Dig my blog again to find out libatlas packages can account for this, and only a re-compilation of ncs is necessary. However, I am not sure how much time it could cut down. You could try.

    $ sudo apt-get install libatlas-headers libatlas-base-dev

    Good luck :)

    Best regards,

    ReplyDelete
  17. Salad-
    OK. I have Code_Saturne running mostly like I want it to, but not quite- I have Atlas for Blas, but have no way of knowing if it is doing anything for me.
    I have encountered an issue with multiprocessors- the particular problem I am running runs fine with a single processor, and after figuring out how to activate the openmpi features (nice help screen from Saturne- thanks), the case crashes very quickly...I see in the forums others having problems with more than 2 processors with newer versions of openmpi, Slowly, I am getting to the point where I was with CAELinux before!
    I have CFDSTUDY, but have had problems trying to build it as well- I will try again later. For now, I don't have a problem with running them separately...
    First impressions after getting through the build issues is that the beta2 is a pretty good improvement over what I was using with CAELinux...

    ReplyDelete
  18. Salad-

    Thanks for the instructions. I installed on OpenSUSE 11.2 64bit and Salome without to much problems...

    When trying to run the pre-processor I get the following:

    error while loading shared libraries: libhdf5.so.0

    libhdf5.so.0 is a link to libhdf5.so.0.0.0 in Salome and I did specify the /lib directory containing the file when installing.

    How can I see where cs_preprocessor looks for the file so I can put a link there?

    Cheers,
    k

    ReplyDelete
  19. Hi Karl,

    I suppose we need to specify the hdf library location by

    :/$ ./configure --with-hdf5=/home/salad/salome_5.1.3/hdf5-1.6.9

    You can replace the directory path with yours.

    If you already did this, I suppose that libhdf5.so.0 can be found but couldn't be loaded for some reasons. Can you get more information about the error?

    Can you also try

    :/$ sudo ldconfig
    :/$ sudo updatedb

    By the way, did you use the wizard version to install SALOME or use the Universal binaries package?

    Good luck!

    Best regards,

    ReplyDelete
  20. Thanks, everything works well now.

    I just included a link in /usr/local/lib to libhdf5.so.0.0.0 in the Salome install. So your method works for OpenSUSE 11.2 as well! Very easy install.

    ReplyDelete
  21. Hi Karl,

    Glad to know that everything works now. Compiling Code_Saturne with SALOME is easier since the libraries can be re-used. Thanks for letting me know that works on OpenSUSE as well.

    Best regards,

    ReplyDelete
  22. Hy,

    I'm trying to install CS-2.0.0 on a RedHat Linux x86_64 system. I install almost all modules, but I have a problem with the ncs module. I get this error at make:

    make[3]: Entering directory `/opt/apps/saturne-aerospace/src/ncs-2.0.0-beta2/gui/Base'
    Traceback (most recent call last):
    File "../../gui/sbin/mkpyqt.py", line 16, in
    import PyQt4.QtCore
    ImportError: /opt/apps/saturne-aerospace/lib/python2.5/site-packages/PyQt4/QtCore.so: undefined symbol: PyUnicodeUCS4_AsLatin1String

    All the packages are install in the same folder /opt/apps/saturne-aerospace. I tried to google the error, but I haven't found anything. Any idea?

    Thanks,
    Marius

    ReplyDelete
  23. PS: sorry for my english but i'm quite tired :P

    ReplyDelete
  24. Hi Marius,

    I only experienced RedHat a long time ago :( I try to suggest.

    Firstly I suggest trying CS 2.0-rc1 instead. I use Python 2.6.5 on my Ubuntu. If there's a way to update your python 2.5, please try it.

    The error seems to be a unicode issue. It indicates that your Qt unicode is using a different byte-size for unicode characters than the python your CS is running from. A possible solution is re-install PyQt or even compile to install it manually. By the way, what is your system language?

    Good luck :)

    Best regards,

    ReplyDelete
  25. Hi,

    Finally I managed to install CS 2.0-rc1. I solved the unicode problem by specify at python installation "./configure --enable-unicode=ucs4". I had another problem with the qt4 version (the version on my redhat distribution has qt4 4.2.1 and cs requires a qt4 version >= 4.3.0), but I solved it by using the qt4 version from salome :)

    Thanks for the help.


    Marius

    ReplyDelete
  26. Hi Marius,

    Good. Glad to know that. This is also a good hint for others who might encounter similar problems.

    Best regards,

    ReplyDelete
  27. Thank you for this fascinating post, I am happy I observed this website on Google. Not just content, in fact, the whole site is fantastic. Binary Today

    ReplyDelete