Message processing and thread safe containers

Often you want to have an application that’s driven by external events such as messages sent from another machine. The best way to do this is to have one thread that receives and stores the messages in a container while the main thread is takes them off of it and process them.

The benefit of this approach is that you are not risking to lose messages if the processing takes too long or coincides with a messages being received, however you still have to be careful not to allow the simultaneous access to the container.

The STL containers are not thread safe, so you have to roll your own. The key point is to keep it simple – you want to have a way to add and remove from the container that ensures that only one thread at a time is accessing the container.

Of course if you are writing a thread safe container, you want to make it generic so you can re-use it:

#include <queue>
#include "boost/thread/thread.hpp"

template <class DataType>
class BlockingQueue
{
   private:
      //Lock on the mutex to ensure only
      //one operation is performed at a time
      boost::mutex threadSafety;
      std::queue<DataType> messageQueue;
   public:
      void add(DataType& newMessage)
      {
         //Prevent other calls to add or remove from executing
         //for the duration of this function
         boost::lock_guard lock(threadSafety);
         messageQueue.push(newMessage);
      }

      DataType remove()
      {
         //Prevent other calls to add or remove from executing
         //for the duration of this function
         boost::lock_guard lock(threadSafety);
         return messageQueue.pop();			
      }
      bool isEmpty()
      {
         boost::lock_guard lock(threadSafety);
         return messageQueue.empty();
      }
      void clear()
      {
         boost::lock_guard lock(threadSafety);
         while (!messageQueue.empty())
         {
            messageQueue.pop();
         }
      }
}

Notice that each method first locks the mutex to ensure that only that function is accessing the queue. The lock guard will lock the mutex at the time of it’s creation and will release it once it goes out of scope. If another thread has the lock on the mutex, the thread will wait until it can get access.

Now let’s figure out the threading for the message handling. There will be one thread to read the messages and put them in the queue and another to process them.

//Some class that holds the messages:
class Message;

//Forward declarations details will follow later
void readMessageFunction(BlockingQueue& blockingQueue);

using namespace boost;
int main()
{
   //Create the queue
   BlockingQueue<Message> msgQueue;
   //Start the receiver thread
   thread readMsgsThread(readMessageFunction, ref(msgQueue));

   unit numberRetries = 0;
   //Fetch a message from the queue and process it
   while (true)
   {
      if(msgQueue.isEmpty())
      {
         numberRetries++;
         this_thread::sleep(boost::chrono::miliseconds(50));
      }
      else
      {
         Message incomingMsg = msgQueue.getMessage();
         if (incomingMsg.getMsgCode() == SHUT_DOWN)
         {
            break;
         }
         //Do whatever you want to do with the messages
         processMessage(incomingMsg);
      }

      //Timeout - exit the main loop and terminate
      if (numberRetries >= MAX_NUM_RETRIES)
      {
         break;
      }
   }// end while
   return 0;
}

The readMessageFunction takes a reference to the container class and is then started in a new thread. It will monitor the source of the messages (socket or whatever) and will insert them in the queue as they arrive.

At the same time the main thread checks the queue every so often and processes any messages it finds in it.

The way I implement the locking in the example above has a potential problem, namely that while I check if the queue is empty before I remove from it, the lock is released between the end of the check and the actual remove. If there were multiple threads reading from the queue this would be a problem since one thread could remove a message from the queue right at that moment causing an attempt to remove from an empty queue.

In this case it doesn’t make much sense to have multiple readers removing from the queue. This is often the case, unless you are implementing some sort of shared resource pool, but having to write something like that is rare.

So there you have it – a message handling implementation completed with generic thread safe container that you can easily modify to if you want something a bit more fancy – a priority queue perhaps.

Simple data sharing between theads

Multi-threading allows you to execute portions of your code concurrently. While on a single CPU the benefits of multi-threading are limited, multi-core systems allow true concurent execution, thus reducing the time it takes for the task to finish.

The basic threading is easy, you just declare a function and run it in thread.

#include <thread>
#include <iostream>
using namespace std;

void complicatedCalculation1()
{//Do stuff here}

void complicatedCalculation2()
{//Do stuff here}

int main()
{
   //Creates a thread and starts it
   thread calc1Thread(complicatedCalculation1);

   //Creates a thread and starts it
   thread calc2Thread(complicatedCalculation1);

   //Wait for each thread to finish
   calc1Thread.join();
   calc2Thread.join();

   cout<<"All done"<<endl;
   return 0;
}

Continue reading “Simple data sharing between theads”

MS SQL Server cyclical reference nonsense

Most relational database software allows a change in one table to trigger changes in other tables based on a foreign key relation. For example let’s say we have two tables: Users and Orders that have a foreign relation defined on the user id field.

Users      Orders
---------  --------
USERID     ORDERID
USERNAME   USERID

SQL Server allows you to define cascading deletes so that when you delete an entry from the users table, all related entries in the orders table are also going to be deleted. Continue reading “MS SQL Server cyclical reference nonsense”

Microsoft got cool? Apple is getting clobbered in the ad wars?

The new Iphone 5 has been out for some time now. The ads are back on the air – advertising the Iphone’s groundbreaking new features – the shape of the headphones! Who else but Apple would’ve thought that ear buds have to match the shape of the earlobe? And that you can use your thumb to  scroll your smart phone’s screen? Those are breathtaking innovations! I guess the 2% R&D budget is paying off…

For sure, the sales are through the roof, but Apple’s monopoly on cool is starting to slip. Samsung are clobbering them with their Samsung Galaxy S3 ads; the message is clear – the cool kids have moved on. Samsung is doing to Apple, what Apple did to Microsoft with the Mac vs PC commercials! And if Apple keeps pushing the same old products, it just might work!

Speaking of big old Microsoft – they got cool! I’m talking about the Surface table commercial that’s playing on TV. Snapping keyboards, lively, colorful, young people jumping around, dancing … The first few times I saw it, I didn’t even realize it’s a Microsoft commercial.

Whoever came up with that commercial has to be given an award. Compared to the weird and creepy Bill Gates & Seinfeld commercial, this one is an incredible improvement. I mean, who in their right mind will put Bill Gates in a commercial? Actually I know who – Apple.

I’m glad for them. Finally, 5 years after they introduced the multi-touch Surface table computer, Microsoft are doing something with the technology. The tablet look pretty good, but they have to get the cool factor working for them.

That’s extremely difficult. Microsoft is synonymous to uncool. They are not helping themselves either – copying everything from everyone, they’ve become target for ridicule  They even cloned Steve Jobs for the introduction of this very same Surface tablet!

I sure hope this new ad is a break with practice.

Google’s gtest not handling friend tests correctly

I was using Google’s gtest unit test framework today and I found an annoying problem – you can’t properly friend test methods that are in a different namespace than the class being tested.

Here is a typical situation:

namespace MyNamespace
{
   class MyClass
   {
      public:
         MyClass(int inData): data(inData){}
      private:
         int data;
   };
}//end namespace

namespace MyNamespace
{
namespace TEST
{
   class MyTestClassFixture
   {
      static const int fixtureData;
   }
   static const MyTestClassFixture::fixtureData = 5;

   TEST_F(MyTestClassFixture, TestEqual)
   {
      MyClass testClass(5);
      ASSERT_EQ(fixtureData, testClass.data);
   }
}
}

Continue reading “Google’s gtest not handling friend tests correctly”

Oracle C++ API non-sense

I spent 2 hours trying to figure out this nonsense. Here is the problem:

Environment *env = Environment::createEnvironment();
Connection *conn = env->createConnection("username", "password",
connectionString);
string sqlQueryText = "UPDATE myTable SET field4 =:p4, field3 =:p3
WHERE field2 :=p2 AND field1 :=p1";
Statement* updateStatement = conn->createStatement(sqlQueryText);

updateStatement.setInt(1, field1Var);
updateStatement.setString(2, field2Var);
updateStatement.setInt(3, field3Var);
updateStatement.setString(4, field4Var);

updateStatement.executeUpdate(conn);

Continue reading “Oracle C++ API non-sense”

Guard you privacy

In the first article of this series, I went on a pretty long rant about how people are not using secure passwords, how that could lead to their information being stolen, etc, etc.

In this one, I’ll go over what I’m using for my own online security. It’s all based on free software and services,which have been around for years and everyone probably knows about, but since I have not seen any comprehensive tutorials on how to do this, I’ll try to explain in details. Feel free to skip ahead if you think I’m wasting your time, you are not interested in the functionality I describe or whatever.

So enough beating around the bush…
Continue reading “Guard you privacy”

Фонетичен Дворак за български

Когато реших че е крайно време да се науча да пиша по 10-то пръстната система, се разрових в Интернет и намерих системата Дворак. Според Интернет (а всички знаем че Интернет знае всичко и е безгрешен), Дворак се учи по-лесно, скороста на печатане е по-висока, по-ергономична е, намалят стреса на пръстите – абе с две думи само където не ти поднася закуска в леглото.

А може и да има такава опция, но да не съм я намерил още…

Научих Дворка и съм доволен. Има обаче един проблем. Никъде не можах да намеря фонетична Дворак подредба за Български език. Да седна да уча БДС клавиатурна подредба, не ми се занимава, а фонетичната която идва с Уиндоус следва АНСИ подредбата…

Този проблем ме мъчи от доста време, така че най-накрая реших да създам една фонетична Дворак подредба. Ето го резултата.
Continue reading “Фонетичен Дворак за български”

Mars rover Curiosity set to land tonight

NASA’s new Mars rover Curiosity is scheduled to land on Mars tonight.  The best part is that it will be broadcasted live on NASA TV, starting 23:00 (11pm) Eastern time.

Of course, every landing on Mars is exciting, but this one in particular will be spectacular.The previous two rovers, Spirit and Opportunity, which spend over 5 years exploring the Mars’  surface, used a “conventional” landing – after they entered the atmosphere, parachutes were deployed to slow the decent down and the impact of the touch down was handled by a high tech bouncing ball which wrapped the rovers. After the ball stopped bouncing, it opened up and the rover rolled out.

Curiosity however is the size of a car and too heavy to do that, so NASA devised a completely new procedures – which they could not even test completely on Earth. The re-entry will be pretty much the same, but once the heat shield is discarded and parachutes deployed, the landing craft is supposed to fire it’s rocket engines, hover above the ground, open it’s cargo bay door and lower the rover down on a rope. After the rover is deployed, the hover will fly away and crash away from the rover.
Continue reading “Mars rover Curiosity set to land tonight”

Guard you privacy

In “Hackers”, The Plague told his employer that the four most often used passwords are love, sex, secret and god. Not much has changed. Most peolpe still seem to use simple words as passwords, or at the best, combination of their initials and their birth or wedding dates. The “power users” tend to go for 133t sp3@k – which is probably even worst because there are 133t sp3@k dictionaries that can be used for a brute force attack.

Just a few days ago, the media reported that LinkedIn has been hacked and over 6 millions passwords were stolen. Guess what the most common ones were?? Jackpot – god, ilove (I guess times are changing …), sex, 1234 …
Continue reading “Guard you privacy”