Wednesday, January 18, 2017

64-bit Software Development on IBM AIX

In this post I'll talk about software development on IBM AIX by means of open source software tools in concert with native AIX development tools.

Using GCC as the compiler to compile your application in AIX is just fine. However, GCC's ld (ld-gcc) linker is not suitable to be used as the linker. This is because linking in AIX is rather tricky and apparently only AIX linker (ld-xlc) work reliably. You can read more about this issue at Using the GNU C/C++ compiler on AIX and AIX Linking and Loading Mechanism.

AIX also has its set of binary utilities (binutils) programs. They are basically the analog of GCC binutils. AIX has native ar archiver, native ld linker (this one is the linker from the AIX xlc compiler suite), and dump utility which is analog to objdump in GCC binutils.

Now, let's see what you need to do to create 64-bit applications in AIX by using the GCC compiler and the native binutils.

  • Pass -maix64 parameter to the GCC C compiler to instruct it to emit the correct AIX 64-bit object file that can be handled by the native AIX linker.
  • Pass -b64 parameter to the native linker via GCC. You should use GCC's -Wl switch for that. The overall parameter becomes -Wl, -b64. IBM AIX ld command reference explains the parameter in detail.
  • Pass -X64 parameter to the native ar archiver to build a 64-bit AIX library. IBM AIX ar command reference explains the parameter in detail.
Once you have build the 64-bit executable or library, you may want to examine it. In Linux or BSD Unix, you would use objdump for that. In AIX, you can use the native dump utility. You need to pass -X64 parameter to dump to instruct it to work in 64-bit mode, i.e. treat the input executable/library as 64-bit binary. For example, the command to show the dependencies of a 64-bit AIX application is: dump -X64 -H.  Refer to the IBM AIX dump command reference for more details.

Listening to Multicast Netlink Socket

Netlink is the recommended way to communicate with Linux kernel from user-space application. In many cases, the communication is unicast, i.e. only one user-space application uses the netlink socket to communicate with a kernel subsystem that provides the netlink interface. But, what if the kernel subsystem provides a multicast netlink socket and you want to listen to the multicast kernel "message"s through netlink? Well, you could do that by bind()-ing to the multicast address provided by the kernel. I'm not going to provide a complete sample code. Just the most important code snippets.

First, you should head over to this netlink discussion to get a sense of the overall netlink architecture.

Once you grasped the netlink architecture, you may follow these steps/algorithm to "listen" to the multicast address(es) provided by the kernel subsystem through netlink:
  1. Init a netlink socket to the kernel subsystem you wish to access. Remember to use #include<linux/[subsystem_header].h>.
  2. Carry-out initialization on the socket if needed.
  3. Bind the socket to the multicast address provided by the kernel subsystem. The multicast address is basically a combination of the following: 
    • The netlink address family, i.e. AF_NETLINK.
    • The netlink multicast group which you can find in the kernel header. For example: the multicast group address for audit subsystem (a constant), is in the audit header file, i.e. <linux/audit.h>
    • Your application's Process ID (PID). 
  4. Read from the socket, when there is data coming in. You might want to use event-based library here, such libev or libevent. In many cases, the kernel only provides a multicast "Read-Only" channel, i.e. you can only read from it. It's not meant to be used to "write" to the kernel.
Step 3 above is probably rather vague. The code below clarify the multicast address that I talked about in that step. Look at the s_addr variable in the code below, it is the multicast address used by bind() to listen to kernel messages. The PID is included in the multicast address because the kernel need to know to which process the message should be sent.
 // ..  
 struct sockaddr_nl s_addr;
 memset(&s_addr, 0, sizeof(s_addr));
 s_addr.nl_family = AF_NETLINK;
 s_addr.nl_pad = 0;
 s_addr.nl_pid = getpid();
 s_addr.nl_groups = AUDIT_NLGRP_READLOG;

 retval = bind(fd, (struct sockaddr *)&s_addr, sizeof(s_addr));
 if (retval != 0) {
  PRINT_ERR_MSG("Failed binding to kernel multicast address");
  return -1;
 // ..
Anyway, because the channel used by the code is multicast channel, multiple user-space application can "listen" to the same kernel subsystem simultaneously. The scenario explained here is not the norm. But, some use cases required this approach.

Monday, January 9, 2017

The Importance of C/C++ Program Exit Status in Unix/Linux

The return value from main() in C/C++ programs a.k.a exit status is often overlooked by less advanced Unix/Linux programmers. Nevertheless, it's important to keep in mind the exit status of your C/C++ code because it will help in the long run. There are at least 2 scenarios where exit status is important:

  1. When you're using shell script to automate processing by using several programs to perform "sub-tasks". In this case, the shell script--very possibly--need to make logical decision based on your program exit status. 
  2. When your C/C++ program is part of a multiprocess program in which your C/C++ program is called/executed (a.k.a fork-ed and exec-ed) by the parent process. In many cases, the parent process need to know whether your program executes successfully or not.

Now, let's be more concrete. Let's say you anticipated that your C/C++ program will be invoked by bash-compatible shell. In that case, your code's exit status must make sense to bash. Therefore, you should:

Following the rules above doesn't necessarily mean your C/C++ program will be bug free because some of the exit status are ambiguous. Nevertheless, it should make it more palatable to be combined into larger system and ease debugging.

As closing, let's look at a very simple code that uses sysexits.h below.
#include <stdio.h>
#include <sysexits.h>
 * Usage: 
 *  test_code  -x param1 -z param2
int main(int argc, char *argv[])

 if (argc != 5) {
  printf("Usage: %s -x param1 -z param2\n", argv[0]);
  return EX_USAGE;

      //... irrelevant code omitted

       return EX_OK;

Wednesday, December 7, 2016

How to "interface" Runtime Error Handling between C++11 (Modern C++) and C Code

This post explore the idea to interface runtime error handling between C++11 and C. This is important because most OS API are exposed via C libraries. Moreover, there are countless C libraries out there that uses C-styled runtime error handling as well.

Anyway, before moving further I want to emphasize that assertion has a different goal compared to runtime error handling. Assertion is meant to be used to catch logic error in your code during development, before the program/executable is released/used in operational environment. Therefore, this post doesn't concern the use of assertion. This post will focus on runtime errors caused by "invalid" state of system resources, such as non-existing file, failure to allocate heap memory, etc. Preliminary information on when to use assertion can be found over at MSDN: Errors and Exception Handling (Modern C++). The MSDN article is rather centered on Microsoft-platform. But, the principles explained in it are applicable to any C/C++ code.

Lets get back to the main theme: runtime error handling interface between C++11 and C code. MSDN provided a sample solution to the problem as well: How to: Interface Between Exceptional and Non-Exceptional Code. Unfortunately, the sample provided by MSDN still doesn't use C++11 smart pointer to manage the file HANDLE resource that it uses. Moreover, it's Windows-centric. Therefore, it's not a "pure" C++11 solution yet. However, the idea presented by the MSDN sample is profound and has been adopted in the code for my previous post about using custom deleter.

The basic idea for runtime error handling in C and C++ is different:
  • In C, you have the errno variable from the standard C library or if your code runs in Windows you can query the error code via GetLastError(). Because I'm trying to be platform independent, let's focus on using errno. In C, your code checks the value of the errno variable after a call to a C library function to check for runtime error. Side note: This C runtime error approach is akin to "side-band" signaling in hardware protocol because you don't get the full picture from the return value of the called function. Instead, you need to check other variable via different means. 
  • In C++, your code should be using exception as the mechanism to propagate error "up-the-stack" until there is a "handler" that can handle the error. If the error is unhandled, the standard behavior is to call std::terminate which normally terminate the application.
Looking at the two different mechanisms for runtime error handling in both C and C++, you must have come up with the answer: wrap the C error code into C++ exception.  I provide a sample code that shows how to wrap C error code into C++ exception at's an updated version of my C++11 custom deleter sample code. Feel free to clone it. The rest of this post explains the code in that Bitbucket URL.

The steps to wrap C error code into C++11 exception are:
  1. Create an exception class that derives from runtime_error class.
  2. Store the error code/number in that exception class.
  3. Create a method in that exception class that transforms the error code into human readable error message (string).
  4. Throw an object of the exception class type in places where a runtime error might occur.
  5. Catch the exception in the right place in your code. 
The preceding steps are not difficult. Lets examine the sample code in more detail to understand the steps.

The exception class is FileHandlerException. It's defined as follows:
class FileHandlerException : public runtime_error
    explicit FileHandlerException(int errNo, const string& msg):
        runtime_error(FormatErrorMessage(errNo, msg)), mErrorNumber(errNo) {}

    int GetErrorNo() const {
        return mErrorNumber;

    int mErrorNumber;
The FileHandlerException class is derived from runtime_error class--the latter is part of stdlibc++ in C++11. The FileHandlerException class uses its mErrorNumber member to store the error code (errno value) obtained from C. The FormaErrorMessage() function is a custom function that transforms errno value into human-readable string via the C strerror() function. This is FormaErrorMessage() function implementation:
string FormatErrorMessage(int errNo, const string& msg)
    static const int BUF_LEN = 1024;
    vector<char> buf(BUF_LEN);
    strncpy(, strerror(errNo), BUF_LEN - 1);

    return string( + "   (" + msg + ")   ";
As you see, it's not difficult to implement the wrapper for C runtime error code. Lets proceed to see how the exception class is being used.
explicit FileHandler (const char* path, const char* mode)
        mPath{path}, mMode {mode}
        cout << "FileHandler constructor" << endl;

        FILE* f = fopen(path, mode);

        if (f != NULL) {
            unique_ptr<FILE, int (*)(FILE*)> file{f, closeFile};
            mFile = std::move(file);

        } else {
            throw FileHandlerException(errno, "Failed to open " + string(path));

    } catch (FileHandlerException& e) {

        throw e;
In the preceding code, if the call to fopen() failed to produce a usable FILE object, an exception object of type FileHandlerException is initialized and thrown. The catch part of the code simply re-throw the exception object higher-up the stack. The code that finally catches the exception object is shown below.
try {
        DeleterTest::FileHandler f(argv[1], "r");
        //.. irrelevant code omitted
    } catch (DeleterTest::FileHandlerException& e) {

        cout << "Error!!!" << endl;
        cout << e.what() << endl;
        cout << "errno: " << e.GetErrorNo() << endl;
The final "handler" of the exception object simple shows the error string associated with the runtime error, i.e. what causes the failure to obtain a valid FILE pointer.

One final note about exception support in C++11: There is no comprehensive support for Unicode character set yet. I've looked up the web for explanation on the matter but all of them have the same conclusion. Please comment below if you know better answer or update to the problem.

Hopefully, this post is useful for those doing mixed C and C++ code development.