Thread Loop Classes for Server Apps

==============================================================================
C-Scene Issue #3
Thread Loop Classes for Server Apps
Chad Loder
==============================================================================

Note: All my examples will use the Win32 API, but it should not be too hard to generalize to POSIX. I assume some familiarity with the Win32 API, especially its multithreading functions. If you are not familiar with these functions, I suggest you read Richter's Advanced Windows books.
I also assume that you are familiar with C++ objects (including virtual and pure virtual functions, constructors and destructors, inheritance, etc.). I have also, for pedagogical reasons, not included bug-catching devices like asserts and error condition handling. In your code, you should always either handle a condition gracefully, or detect it and notify the user in debug mode.

Finally, some Windows programmers may notice that I have mixed Win32 API functions (like CreateThread) with C Run-Time Library functions - this is also for pedagogical purposes. In real life, I wouldn't.

A common component of a multithreaded server is a threaded-off routine that loops until some event occurs. In this article, I will demonstrate how you can encapsulate this type of routine in what I call a "thread loop class".

A typical concurrent network server might have one routine for each of the following tasks, each running on a different thread:

Process console input
Wait for socket connect requests, calling and accept() on them and passing the new socket to a client i/o routine
Process client I/O (one thread per client)

Of course, this list will vary; some servers treat the console as merely another client connection. Servers with a high level of concurrency might not create a separate thread for each client connection (for performance reasons), and there are usually many other tasks in a given server with this kind of behavior.

All of these tasks will keep looping for input until some event (either external or based on that input) tells them that it's time to stop. Then they will need to shut themselves down gracefully (warning clients, giving clients a chance to disconnect, flushing data to a cache, freeing resources, etc.).

How do we encapsulate this behavior in a class? Well, each thread has many things in common, but each thread also has at least one very different function that loops. This suggests that we create an abstract base class and derive from it for each type of loop routine we want to encapsulate. This loop routine is the one where all the action occurs, the threaded-off one; we will call it the action proc; it will not be implemented in the base class because there is no meaningful default action for a thread loop.

There is one slight complication. The action proc in a derived class will need to have static linkage in order for us to thread it off, because we have to pass its address to the CreateThread() API function. But if we implement the action proc as static, then we will not be able to call non-static functions or access non-static data for the class from the proc, which would in fact make the class useless.

We can get around this difficulty with a little sleight of hand. It will make our class a little uglier, but once implemented, we won't have to think about it any more. We will make the action proc static, but we will pass it a pointer which corresponds to the actual this pointer for the instance of the class (via which we can access non-static data and functions). In the base class, we will implement the Initialize() member, which will call the CreateThread() API function, passing it the address of the action proc, which is returned by a non-static pure virtual member called GetActionProc, which has to be implemented in the derived class. It sounds complex, but it's easy when you look at the code:

#include <windows.h>

// abstract base class
class ThreadLoop
{
    public:

    // constructor
    ThreadLoop()
    {
        m_hQuit = CreateEvent(NULL, TRUE, FALSE, "QuitEvent");
        m_hCleanup = CreateEvent(NULL, TRUE, FALSE, "SuccessQuitEvent");
    };

    // default destructor - clean up and free memory (implemented below)
    virtual ~ThreadLoop();

    // creates the action thread in a suspended state; call Start() to activate
    virtual void Initialize()
    {
        m_hThread = CreateThread (NULL, 0, GetActionProc(),
            (LPVOID)this, CREATE_SUSPENDED, &m_dwThreadID);
    };

    // activates the action thread; must call Initialize() first.
    virtual void Start()
    {
        ResumeThread(m_hThread);
    };

    // stops the action thread permanently by triggering quit event
    virtual void Stop()
    {
        SetEvent(m_hQuit);
    };

    // suspends the action thread until Start() is called again
    virtual void Sleep()
    {
        SuspendThread(m_hThread);
    };

    virtual void QuitOK()
    {
        SetEvent(m_hCleanup);
    };
    // implement in derived class - returns addr of the action proc
    virtual LPTHREAD_START_ROUTINE GetActionProc(void) const = 0;

    protected:

    HANDLE m_hThread;           // handle to the action proc's thread
    DWORD  m_dwThreadID;        // the ID of the action proc's thread
    HANDLE m_hQuit;             // manual event, set when Stop() is called
    HANDLE m_hCleanup;          // set when loop has cleaned up after itself
};

ThreadLoop::~ThreadLoop()
{
    DWORD dwThreadStatus;

    /* if m_hCleanup has not been set, we assume that the server has not been
    notified of a shutdown, so try to notify it ourselves and see what happens
   */
    if (WAIT_TIMEOUT == WaitForSingleObject(m_hCleanup, 0))
    {
        Stop(); // try graceful shutdown by issuing command

        // after calling stop function, wait two seconds for cleanup
        if (WAIT_TIMEOUT == WaitForSingleObject(m_hCleanup, 2000))
        {
            GetExitCodeThread(m_hThread, &dwThreadStatus);
            
            // if the thread is still active, we will murder it
            if (STILL_ACTIVE == dwThreadStatus)
            {
                TerminateThread(m_hThread, 1);
            }
        }
    }

    CloseHandle(m_hQuit);
    CloseHandle(m_hCleanup);
}

There is the framework for the class. It is fully functional; all it needs is the implementation of GetActionProc() and the action proc itself in the derived class. Notice the destructor. If the object goes out of scope before a Stop() command has been issued, the destructor tries to issue a Stop() and wait for the action proc to clean up (the action proc should call QuitOK() on successful cleanup). If cleanup hasn't succeeded after two seconds, we do it the hard way by killing the action proc. You should obviously avoid this situation, because bad things happen when you kill open threads (data corruption, unfreed resources, etc.).

Notice also that most of the member functions in the base class are virtual ( including the destructor). I have done this for two reasons: First, we cannot foresee all the needs of derived classes - they might need to override some of the default behavior; Second, if we are using pointers, we want the appropriate function to be called based on the type of object pointed to, not on the type of pointer we are using. For example, if we do something like this:

    ThreadLoop* theLoop = new DerivedLoop;
    // ...execute some code using this loop
    delete theLoop;

Now, if the destructor in the base class were not virtual, calling delete on theLoop would invoke ThreadLoop::~ThreadLoop(), the base class destructor (because theLoop, although it points to a DerivedConsoleLoop, is of type ThreadLoop*). This would skip the DerivedLoop destructor, which would almost certainly result in memory leaks or other problems. However, since the destructor in the base class is virtual, no matter what kind of pointer you use, the correct derived-class destructor will be called when you delete it.

Now we will derive a class from ThreadLoop. This class will be the console portion of a generic network server. It will process input from the console.

#include <iostream.h>
#include <conio.h>

class Console : public ThreadLoop
{
    public:

    Console() : ThreadLoop()
    {
    };

    ~Console()
    {
        CloseHandle(m_hInput);
    };

    // the action proc, implemented below
    static DWORD WINAPI KeyboardLoop(void* const p);

    // returns the address of the action proc
    LPTHREAD_START_ROUTINE GetActionProc(void) const
    {
        return KeyboardLoop;
    }

    private:

        HANDLE m_hInput;    // stdin
        char m_szInput[256];// user input
};

// implemented as DWORD WINAPI: correct calling convention for CreateThread()
DWORD WINAPI Console::KeyboardLoop(void* const p)
{
    // p is the this pointer
    Console *thisConsole = (Console *)p;

    // get STDIN, so we can sleep on it while waiting for input
    thisConsole->m_hInput = GetStdHandle(STD_INPUT_HANDLE);

    // little array used for WaitForMultipleObjects()
    HANDLE phObjects[] = { thisConsole->m_hQuit, thisConsole->m_hInput };

    bool bContinue = TRUE;
    DWORD dwWaitReturn;

    while (bContinue)
    {
        cout << "[-Console ready-] ";
        cout.flush();

        // wait either for keyboard input or for a Stop() event
        dwWaitReturn = WaitForMultipleObjects(2, phObjects, FALSE, INFINITE);

        switch (dwWaitReturn - WAIT_OBJECT_0)
        {
            case 0: 
                // Stop() event, so exit the loop
                bContinue = FALSE;
                continue;
            case 1:
                // input from the console, read it
                cin.getline( m_szInput, 255, '\n' );
                // you will obviously want to parse this in real life 
                if (0 == strcmp(m_szInput, "quit"))
                {
                    bContinue = FALSE;
                    continue;
                }
        }
    }
    
    cout << "Shutting down console" << endl;
    thisConsole->QuitOK();
    return 0;
};

Now we can invoke a simple console server with the following code.

int main(void)
{
    Console theConsole;
    theConsole.Initialize();
    theConsole.Start();
    Sleep(INFINITE);    // this is not good!
    return EXIT_SUCCESS;
}

In real life, you would not put the main thread to sleep this way, because it would have no way of waking up when the user quit from the console (not to mention that it would waste resources, albeit only one thread). There are a couple ways around this problem that suggest themselves. The first way is perhaps the easiest: implement a member function which returns the cleanup event handle from the class, then have main() wait forever on that handle:

HANDLE Console::GetCleanupHandle() const
{
    return m_hCleanup;
}

int main(void)
{
    Console theConsole;
    theConsole.Initialize();
    theConsole.Start();
    WaitForSingleObject(theConsole.GetCleanupHandle(), INFINITE);
    return EXIT_SUCCESS;
}

The second possible solution is more involved, but perhaps more robust in the long run. For every derived ThreadLoop class you make, implement a constructor where you pass in a handle which corresponds to a program-wide quit event handle. All loops will wait on this event (in addition to other handles) - whenever this event is triggered, they begin to clean up and shut down. In classes like Console, when the user types "quit", trigger this global event handle and all the rest of the code will start to clean up. The main() function will wait on this event handle - after it's triggered, main() will sleep on the cleanup handle of the object which takes the longest time to clean up, or it will sleep for a certain safe amount of time (like 5 seconds) until everyone cleans up. Depending on how you organize your program, you can implement this more robustly, so that main() doesn't exit until every loop in an entire collection (you could have a linked list of ThreadLoop pointers) has cleaned up. The possibilities are endless.