Compile OpenCASCADE Community Edition the Easy Way! (Update)

Introduction:

After a very late last night as a result from the following: splitting my head for three hours on Finite Element Modelling and for some reason watching Back to the Future:Part II whilst figuring out some code for the section plane, unbelievably I managed to wake up reasonably fresh.

Before starting on the chore of further University work this afternoon. I've been playing with compiling OpenCascade Community Edition in attempt to remove the shackles of Ubuntu's dictated policy of offering old packages.

All of this was inspired by a message on IRC #freecad over the past week about compiling OpenCascade with debug flags, so we actually have useful output when OpenCascade crashes without a warning (mostly from an unhandled exceptions on our part).

Benefits:

• The other benefits are testing some new features - only recently has a commit been made to improve performance of boolean operations that will soon be merged - see discussusion
• Full debug build so we can trace crashes correctly
• Living life on the edge of your seat!

The first compile went really well, however, after attempting to compile FreeCAD I found that there were a few missing libraries that prevent FreeCAD from building. These were the visualisation and Model Exachange packages used by SMESH and Part for the IGES and STEP formats import.  I am not sure if these should have been detected by FreeCAD's cmake but anyway, trial and error prevails.

I also attempted to build OCE using TBB which is a threading library that can be used to speed up operations, however from what I read this is only used in a few algorithms.

Note:

Compiling OpenCascade does take a long time, around probably 50% longer than FreeCAD for a clean build. On my six core AMD Bulldozer computer, it took in the region of 18 minutes in total to compile with each core at 100%. Also remember that you will need additional time to compile FreeCAD again. Once this has been done, recompiling is very quick.

Be prepared to find something useful to do in the mean and drink tea or even a beer for a good half an hour.

Compiling OCE OpenCascade Community Edition (Easy Guide):

Reminder: expect 20-30 Minutes to compile:

Clone the repository:

Grab the latest git source from the official Open Cascascade Community Edition from the github page. The source is around 240mb when fully expanded and may take a while to download on a slow internet connection.

git clone https://github.com/tpaviot/oce.git

Create a build directory:

Create a suitable build directory, this can be in the main source folder but preferably outside to prevent potential conflicts

mkdir oceBuild && cd oceBuild

Install libftgl-dev in Package Manager:

You need to install following development package for libftgl to compile the Visualisation Package in OCE. This will depend on your Distribution. 

Install libtb2 & libtbb-dev in Package Manager:

For enabling TBB - Threaded Building Blocks Library to be used in multithreading, you will need to ensure that these are installed on your system. Below are the package names in Ubuntu.

Remove any previous Opencascade libraries provided by distro:

I don't think OpenCascade and OCE can be run in parallel on the same machine - correct me if I'm wrong. Therefore you will need to remove any of these on your system.  

Configure the sources using CMake

Configure the OCE sources running the following cmake command in the current directory. This will install OCE to your /usr directory. 

cmake -DCMAKE_INSTALL_PREFIX:PATH=/usr/ \
      -DOCE_INSTALL_PREFIX:PATH=/usr/ \
      -DOCE_WITH_FREEIMAGE=OFF \
      -DOCE_WITH_GL2PS=OFF \
      -DOCE_VISUALISATION=ON \
      -DOCE_DATAEXCHANGE=ON \
      -DOCE_MULTITHREAD_LIBRARY=TBB \
      -DCMAKE_BUILD_TYPE=Debug \
      ../oce

If you're installing on Ubuntu or Debian based distribution, it's advisable to change the install directory to /usr/local to isolate user compiled packages (@yorik) - (I need to confirm if FreeCAD can detect the location of the OCE header files from the location)

-DCMAKE_INSTALL_PREFIX:PATH=/usr/local \

You may find that there are some missing development dependencies, so if there are you will have to install these using your package manager. I found generally if you could install FreeCAD you wouldn't experience any problems atleast on Ubuntu.

Create a release build (Improve Performance):

You may not want debug flags enabled so that you can have full compiler optimisations and full performance when using FreeCAD: change the line previously from

DCMAKE_BUILD_TYPE=Debug \

to

DCMAKE_BUILD_TYPE=Release \ 

Begin Compiling:

When you start compiling OCE  be prepared to wait a while. Replace 6 with the number of cores or independent threads that can be run on your computer to speed up compiling by running concurrent make operations.

make -j6 -pipe

There has been some discussion whether the '-pipe' argument can reduce compiling times. If someone has enough time to make a comparison with OCE it would be interesting to see the difference.

Install:

Usually I install any compiled software to my user directory, but that wouldn't be straightforward. Therefore CMake was configured to install to /usr directory for simplicity later on. 

sudo make install

Installing on Debain / Ubuntu Systems:

Another helpful hint from @yorik is to install using the built in utility checkinstall. This provides a convenient method of uninstalling OCE using the system's package manager. Once you have finished compiling, run the following command:

sudo checkinstall

Type (y) to accept the defaults, then you will be presented with a menu to further describe the package - do this at your own leisure:

Once checkinstall is completed - this will take a while, it will produce a .deb package and OCE will have been installed to your system and registered in your system package manager.

Recompile FreeCAD:

There will be changes to the development sources since we are using a different edition of OpenCascade with FreeCAD. You will need to clean all the sources. From your previous FreeCAD build directory make sure run the following before recompiling (see previous post on compiling FreeCAD :

sudo make clean

If OCE was compiled and installed correctly, whilst configuring FreeCAD you will find the following log message (OpenCASCADE Community Edition has been found):

Recompile once again and you should have FreeCAD working with the latest OCE!

Hopefully people will have understand the guide and will now have a working version of OCE that is is used by FreeCAD. If there seems to be problems or something isn't clear, please let me know and I will update the guide. 

Debugging C++ using KDevelop

This is leading on from another post. Without being able to debug a program, it's very difficult to get anywhere or learn mostly from stupid mistakes and is a quintessential for making progress. Thankfully having a good IDE makes all the difference and I present to you how I use KDevelop to debug FreeCAD during development.

Debugging with KDevelop:

First ensure that you are compiling with debug symbols / information and set up KDevelop so that you can compile and launch FreeCAD successfully. Please see my previous post for information.

Open up KDevelop and then you can hit Debug.

The KDevelop workspace will now change to Debug Mode. It is more convenient to present tool panels that are more relevant to debugging than programming. The only frustration is that the Editor will not remember which file you have currently opened from the normal mode.

You can customise which tool panels are available by right clicking and selecting like so:

Useful tool panels for debugging are:

• Variables
• Breakpoints
• Frame Stack

Creating and using breakpoints using KDevelop:

FreeCAD will load, but can take slightly longer since the GDB debugger is launched and use to monitor the FreeCAD executable. Breakpoints allow you to control the flow of program at any point in time where you can:

• Pause on a line / statement (A statement is usually terminated by a semicolon, e.g. int i = 0;)
• Run each statement one by one
• Continue to the next breakpoint

By setting breakpoints, it allows the programmer to control to examine segments of code and check the behaviour of the program or complex algorithms. 

Setting a Break Point:

In the code editor window, look for an area of code that you want to examine and isolate a line where you want to pause the execution of the program. In KDevelop, click on the furthest area adjacent to the line to create a breakpoint.  KDevelop indicates when a break point is set by displaying a ladybird / ladybug.

Sometimes it is useful to set the editor to visualise the line numbers which can be done by enabling the option at Editor->View->Show Line Numbers

Now, run the program normally and attempt to 'trigger' the break point by performing an action that will cause the segment of code you want to be executed - sometimes that's not always straightforward! When a break point is hit, KDevelop will highlight this.

We have paused the execution of the program. We can control the flow by using the debug toolbar at the top.

Importantly, Step Over will run to the next statement on line 43 - this would be std::string path = hGrp...

Continue will run till it reaches the next Break Point, this could be set in the same segment of code, or in another completely different area of the program. So in the example below clicking continue would run all the program till it reaches line 48 - args.push_back(QString...

The other two I rarely use, but they will enter a function or exit a function, but often I found going directly into these function sources and setting my own break points are easier. All break points are shown in the Break Points tool view. These can be toggle on or off and can be deleted. 

*Useful hints:

• Remove all previous break points if you have recompiled FreeCAD with modifications to the code, or even just launching the program. I find it much quicker to launch FreeCAD in debug mode by doing this.
• The more editors tabs you have open, it appears to take longer to launch and consumes more memory.

Making use of Break Points:

When we stop the flow of the program, we can examine the current state of the 'stack'. KDevelop makes this exceptionally easy. We can find the current values of variables or properties of classes/objects. This can be done by highlighting over a variable or object using your cursor:

Alternatively you may use the Variables Tool View to examine all the variables accessible within the 'current scope' or on the stack. It presents a hierarchical tree and can present meaningful data for known variables types and classes within the project tree:

All it takes now is some logic and wit to figure out what is causing the problem. Sometimes it can be blindingly obvious and for other times it can take hours :( The more practice you get the easier it seems to become.

The Frame Stack:

This is most useful when you have a sudden crash.

 Conventionally a stack can be imagine like a stack of  dirty plates at a restaurant. When you enter a new program scope (e.g. run a body of a function) you add on a dirty plate (so the dirty food represents variables). Conversely, exiting a program scope you clean the plate and take off the stack of dirt dishes. I tend to imagine it like an onion, you must peel a layer to access the inner layer. 

You can view the current state of the stack using the Frame Stack Tool View, when a program crashes GDB will display the state of the frame stack to help identify where the crash occurred. This will be at Depth 0, but logically you may have to look higher up to understand why. Usually the current state of the stack variables can also be accessed using the Variables tool view. (*precaution GDB will crash if you attempt to access a null pointer)

Below the current program is on line 49 within the LuxRenderProcess::initialiseSettings method. This method was called in Renderer.cpp on line 363. It is simple as that. 

Hopefully this guide will help you get debugging and make learning how to program c++ within FreeCAD that much easier!

Using KDevelop to develop FreeCAD

Some hard core people bring themselves to use just a text editor, whether its VIM, or a standard desktop editor from Kate, GEdit. These have their places but when you're learning programming or working with large project, it's much quicker to have syntax highlighting/checking and code hinting so that you don't have to trudge through programming documentation. The additional tools for compiling make life much easier too.

One option is to look at the DOXYGEN documentation for FreeCAD. It's seldom used because its both incomplete and it's difficult to navigate without knowing what you're explicitly looking for.

For c++ programming, there are two alternatives, either QtCreator or KDevelop atleast on Linux unless someone can name some viable alternatives. Admittedly QtCreator is very good and is feature complete, for some reason it is incompatible (at least momentarily) importing projects using CMake (a build system). This essentially means you cannot read the source project tree and get useful programming hints, which without I think makes learning FreeCAD much more difficult.

Once you've installed KDevelop, it's very easy to configure the compiler settings. It's in built debugger is extraordinary especially for a newbie to diagnose crashes and above all is built in. The syntax highlighting and code hinting is generally good too. My only dissatisfaction is that sometimes it gets into the habit of crashing, but it's not a problem since it can recover unsaved files.

Here's a quick guide to help you getting it up and running to make your own opinion!

Using KDevelop to develop FreeCAD:

 Firstly ensure that KDevelop is installed and you've got a copy of the FreeCAD source to work with. Load up KDevelop as follows:

Click on Import Project to import the FreeCAD sources.

A dialog will appear. Go to the root directory within the source and then browse for the CMake project file CMakeLists.txt and click 'next'

In the next section you can specify where the FreeCAD sources are built under Build Directory. You can also set the build type. I would always recommend using the Debug build type so that you can easily debug the program for any faults later. Selecting the Release option will provide extra compiler optimisations that will improve FreeCAD's performance, but only significantly in certain areas such as the Sketch Solver. The other two options I wouldn't recommend.

Click OK and KDevelop will start importing the whole project. Further compiler and project settings can be changed under Open Configuration.

If you click under Classes tool bar you can browse the whole source tree which including other library headers. The Flying pink bricks indicate public methods. Icons with Pad Locks are 'private' class members and icons with keys are 'protected methods'.

Open  up a source file (.cpp) and try out the code hinting by pressing CONTROL + SPACEBAR. This will bring up a list of methods below which the class provides. How easy is that?

It can even help suggest the variables, objects that can be passed as arguments into a method:

Make a few little changes and the proceed to click Build Selection. 

This should start configuring your project and it may hint problems. Otherwise normally it will show the build progress in the Build Window as below:

Remember to speed up the build by using multiple cores, append -j4 under the Additional Make Options found in Project->Open Configuration->Make. Compiling usually takes between 15-25 minutes on a good day. However sit back, grab a cup of tea or continuing programming.

Finally Running FreeCAD from Kdevelop

Before running you need to configure the launches. This can be found under Run-> Configure Launches. A new dialog will appear. Choosing the executable FreeCADMain is really straightforward

Now back in the main window you can hit Execute to run normally. Alternativly you can hit Debug to run the application through the GDB debugger. I'll describe how to use the debug tools later!

Hopefully that might help you get started to make tweaks and changes to FreeCAD!

Finding the bottleneck using KCachegrind

I spent the afternoon trying to find a bottle neck within the Scene File Generation for the Render Module. It's not immediately obvious, what was causing the slowdown, which with complicated geometry was making me wait a precious 5 extra seconds. I wanted to write this because if couldn't quickly find on google how to use it to refresh my memory.

Here's a classic case of how Valgrind & KCachegrind came to help. These are virtual machines that profile your code and only need debug information to be compiled. It is both very useful for developers and also importantly testers to find memory leaks, segfaults, 100% cpu loading caused by infinite loops and so forth.

First download both Valgrind and the KDE Graphical Frontend KCachegrind.

Ensure the program your using has debug information and you're not using any compiler optimisation flags (-O3)

Now you can run the Valgrind profiler from terminal in the same directory as the program executable: issueing a command like this.

valgrind --tool=callgrind -v programName

The program will run, but it will be around 20-50x slower than normal during profiling.

-----
For developers if you know the program area that is creating problems, you can dump the output before and after a class method/function e.g.

valgrind --tool=callgrind -v --dump-before=Renderer::generateScene FreeCAD

Ideally the function should be called two times. This will produce 3  seperate files,

1. Output before call
2. Before second call
3. Remaining program till termination

This can make it easier later to diagnose the problem using KCachegrind
-----
Using KCachegrind

Start KCachegrind and then open the log file generated from Valgrind. My first intuition was to look at the Renderer::generateScene where I knew FreeCAD was being slow.

Clicking on this in the left window, will update the right window area showing the call tree.

Scrolling down we can find the culprit (GeomAPI_ProjectPointOnSurf) that is causing the slow down:

This can be seen verified using the source browser:

Looking at the source code and simply commenting this out, I find out that this exert is not actually required. With its removal, generating the scene file was happening near instantly!

I hope this short guide may help some developers and testers!