Message processing and thread safe containers

Often you want to have an application that’s driven by external events such as messages sent from another machine. The best way to do this is to have one thread that receives and stores the messages in a container while the main thread is takes them off of it and process them.

The benefit of this approach is that you are not risking to lose messages if the processing takes too long or coincides with a messages being received, however you still have to be careful not to allow the simultaneous access to the container.

The STL containers are not thread safe, so you have to roll your own. The key point is to keep it simple – you want to have a way to add and remove from the container that ensures that only one thread at a time is accessing the container.

Of course if you are writing a thread safe container, you want to make it generic so you can re-use it:

#include <queue>
#include "boost/thread/thread.hpp"

template <class DataType>
class BlockingQueue
      //Lock on the mutex to ensure only
      //one operation is performed at a time
      boost::mutex threadSafety;
      std::queue<DataType> messageQueue;
      void add(DataType& newMessage)
         //Prevent other calls to add or remove from executing
         //for the duration of this function
         boost::lock_guard lock(threadSafety);

      DataType remove()
         //Prevent other calls to add or remove from executing
         //for the duration of this function
         boost::lock_guard lock(threadSafety);
         return messageQueue.pop();			
      bool isEmpty()
         boost::lock_guard lock(threadSafety);
         return messageQueue.empty();
      void clear()
         boost::lock_guard lock(threadSafety);
         while (!messageQueue.empty())

Notice that each method first locks the mutex to ensure that only that function is accessing the queue. The lock guard will lock the mutex at the time of it’s creation and will release it once it goes out of scope. If another thread has the lock on the mutex, the thread will wait until it can get access.

Now let’s figure out the threading for the message handling. There will be one thread to read the messages and put them in the queue and another to process them.

//Some class that holds the messages:
class Message;

//Forward declarations details will follow later
void readMessageFunction(BlockingQueue& blockingQueue);

using namespace boost;
int main()
   //Create the queue
   BlockingQueue<Message> msgQueue;
   //Start the receiver thread
   thread readMsgsThread(readMessageFunction, ref(msgQueue));

   unit numberRetries = 0;
   //Fetch a message from the queue and process it
   while (true)
         Message incomingMsg = msgQueue.getMessage();
         if (incomingMsg.getMsgCode() == SHUT_DOWN)
         //Do whatever you want to do with the messages

      //Timeout - exit the main loop and terminate
      if (numberRetries >= MAX_NUM_RETRIES)
   }// end while
   return 0;

The readMessageFunction takes a reference to the container class and is then started in a new thread. It will monitor the source of the messages (socket or whatever) and will insert them in the queue as they arrive.

At the same time the main thread checks the queue every so often and processes any messages it finds in it.

The way I implement the locking in the example above has a potential problem, namely that while I check if the queue is empty before I remove from it, the lock is released between the end of the check and the actual remove. If there were multiple threads reading from the queue this would be a problem since one thread could remove a message from the queue right at that moment causing an attempt to remove from an empty queue.

In this case it doesn’t make much sense to have multiple readers removing from the queue. This is often the case, unless you are implementing some sort of shared resource pool, but having to write something like that is rare.

So there you have it – a message handling implementation completed with generic thread safe container that you can easily modify to if you want something a bit more fancy – a priority queue perhaps.

Microsoft got cool? Apple is getting clobbered in the ad wars?

The new Iphone 5 has been out for some time now. The ads are back on the air – advertising the Iphone’s groundbreaking new features – the shape of the headphones! Who else but Apple would’ve thought that ear buds have to match the shape of the earlobe? And that you can use your thumb to  scroll your smart phone’s screen? Those are breathtaking innovations! I guess the 2% R&D budget is paying off…

For sure, the sales are through the roof, but Apple’s monopoly on cool is starting to slip. Samsung are clobbering them with their Samsung Galaxy S3 ads; the message is clear – the cool kids have moved on. Samsung is doing to Apple, what Apple did to Microsoft with the Mac vs PC commercials! And if Apple keeps pushing the same old products, it just might work!

Speaking of big old Microsoft – they got cool! I’m talking about the Surface table commercial that’s playing on TV. Snapping keyboards, lively, colorful, young people jumping around, dancing … The first few times I saw it, I didn’t even realize it’s a Microsoft commercial.

Whoever came up with that commercial has to be given an award. Compared to the weird and creepy Bill Gates & Seinfeld commercial, this one is an incredible improvement. I mean, who in their right mind will put Bill Gates in a commercial? Actually I know who – Apple.

I’m glad for them. Finally, 5 years after they introduced the multi-touch Surface table computer, Microsoft are doing something with the technology. The tablet look pretty good, but they have to get the cool factor working for them.

That’s extremely difficult. Microsoft is synonymous to uncool. They are not helping themselves either – copying everything from everyone, they’ve become target for ridicule  They even cloned Steve Jobs for the introduction of this very same Surface tablet!

I sure hope this new ad is a break with practice.

Oracle C++ API non-sense

I spent 2 hours trying to figure out this nonsense. Here is the problem:

Environment *env = Environment::createEnvironment();
Connection *conn = env->createConnection("username", "password",
string sqlQueryText = "UPDATE myTable SET field4 =:p4, field3 =:p3
WHERE field2 :=p2 AND field1 :=p1";
Statement* updateStatement = conn->createStatement(sqlQueryText);

updateStatement.setInt(1, field1Var);
updateStatement.setString(2, field2Var);
updateStatement.setInt(3, field3Var);
updateStatement.setString(4, field4Var);


Continue reading “Oracle C++ API non-sense”

Mars rover Curiosity set to land tonight

NASA’s new Mars rover Curiosity is scheduled to land on Mars tonight.  The best part is that it will be broadcasted live on NASA TV, starting 23:00 (11pm) Eastern time.

Of course, every landing on Mars is exciting, but this one in particular will be spectacular.The previous two rovers, Spirit and Opportunity, which spend over 5 years exploring the Mars’  surface, used a “conventional” landing – after they entered the atmosphere, parachutes were deployed to slow the decent down and the impact of the touch down was handled by a high tech bouncing ball which wrapped the rovers. After the ball stopped bouncing, it opened up and the rover rolled out.

Curiosity however is the size of a car and too heavy to do that, so NASA devised a completely new procedures – which they could not even test completely on Earth. The re-entry will be pretty much the same, but once the heat shield is discarded and parachutes deployed, the landing craft is supposed to fire it’s rocket engines, hover above the ground, open it’s cargo bay door and lower the rover down on a rope. After the rover is deployed, the hover will fly away and crash away from the rover.
Continue reading “Mars rover Curiosity set to land tonight”