The random ramblings of a techie

Message processing and thread safe containers

Using boost and STL

Often you want to have an application that’s driven by external events such as messages sent from another machine. The best way to do this is to have one thread that receives and stores the messages in a container while the main thread is takes them off of it and process them.

The benefit of this approach is that you are not risking to lose messages if the processing takes too long or coincides with a messages being received, however you still have to be careful not to allow the simultaneous access to the container.

The STL containers are not thread safe, so you have to roll your own. The key point is to keep it simple – you want to have a way to add and remove from the container that ensures that only one thread at a time is accessing the container.

Of course if you are writing a thread safe container, you want to make it generic so you can re-use it:

#include <queue>
#include "boost/thread/thread.hpp"

template <class DataType>
class BlockingQueue
{
   private:
      //Lock on the mutex to ensure only
      //one operation is performed at a time
      boost::mutex threadSafety;
      std::queue<DataType> messageQueue;
   public:
      void add(DataType& newMessage)
      {
         //Prevent other calls to add or remove from executing
         //for the duration of this function
         boost::lock_guard lock(threadSafety);
         messageQueue.push(newMessage);
      }

      DataType remove()
      {
         //Prevent other calls to add or remove from executing
         //for the duration of this function
         boost::lock_guard lock(threadSafety);
         return messageQueue.pop();			
      }
      bool isEmpty()
      {
         boost::lock_guard lock(threadSafety);
         return messageQueue.empty();
      }
      void clear()
      {
         boost::lock_guard lock(threadSafety);
         while (!messageQueue.empty())
         {
            messageQueue.pop();
         }
      }
}

Notice that each method first locks the mutex to ensure that only that function is accessing the queue. The lock guard will lock the mutex at the time of it’s creation and will release it once it goes out of scope. If another thread has the lock on the mutex, the thread will wait until it can get access.

Now let’s figure out the threading for the message handling. There will be one thread to read the messages and put them in the queue and another to process them.

//Some class that holds the messages:
class Message;

//Forward declarations details will follow later
void readMessageFunction(BlockingQueue& blockingQueue);

using namespace boost;
int main()
{
   //Create the queue
   BlockingQueue<Message> msgQueue;
   //Start the receiver thread
   thread readMsgsThread(readMessageFunction, ref(msgQueue));

   unit numberRetries = 0;
   //Fetch a message from the queue and process it
   while (true)
   {
      if(msgQueue.isEmpty())
      {
         numberRetries++;
         this_thread::sleep(boost::chrono::miliseconds(50));
      }
      else
      {
         Message incomingMsg = msgQueue.getMessage();
         if (incomingMsg.getMsgCode() == SHUT_DOWN)
         {
            break;
         }
         //Do whatever you want to do with the messages
         processMessage(incomingMsg);
      }

      //Timeout - exit the main loop and terminate
      if (numberRetries >= MAX_NUM_RETRIES)
      {
         break;
      }
   }// end while
   return 0;
}

The readMessageFunction takes a reference to the container class and is then started in a new thread. It will monitor the source of the messages (socket or whatever) and will insert them in the queue as they arrive.

At the same time the main thread checks the queue every so often and processes any messages it finds in it.

The way I implement the locking in the example above has a potential problem, namely that while I check if the queue is empty before I remove from it, the lock is released between the end of the check and the actual remove. If there were multiple threads reading from the queue this would be a problem since one thread could remove a message from the queue right at that moment causing an attempt to remove from an empty queue.

In this case it doesn’t make much sense to have multiple readers removing from the queue. This is often the case, unless you are implementing some sort of shared resource pool, but having to write something like that is rare.

So there you have it – a message handling implementation completed with generic thread safe container that you can easily modify to if you want something a bit more fancy – a priority queue perhaps.

>>

Simple data sharing between theads

Passing parameters by reference

Multi-threading allows you to execute portions of your code concurrently. While on a single CPU the benefits of multi-threading are limited, multi-core systems allow true concurent execution, thus reducing the time it takes for the task to finish.

The basic threading is easy, you just declare a function and run it in thread.

#include <thread>
#include <iostream>
using namespace std;

void complicatedCalculation1()
{//Do stuff here}

void complicatedCalculation2()
{//Do stuff here}

int main()
{
   //Creates a thread and starts it
   thread calc1Thread(complicatedCalculation1);

   //Creates a thread and starts it
   thread calc2Thread(complicatedCalculation1);

   //Wait for each thread to finish
   calc1Thread.join();
   calc2Thread.join();

   cout<<"All done"<<endl;
   return 0;
}

>>

, , , ,

MS SQL Server cyclical reference nonsense

On SQL Server 2008

Most relational database software allows a change in one table to trigger changes in other tables based on a foreign key relation. For example let’s say we have two tables: Users and Orders that have a foreign relation defined on the user id field.

Users      Orders
---------  --------
USERID     ORDERID
USERNAME   USERID

SQL Server allows you to define cascading deletes so that when you delete an entry from the users table, all related entries in the orders table are also going to be deleted.

>>

, ,

Obama locks in another 4 years

Florida goes to the President

Obama has locked in the win. He has a solid lead in Ohio and Florida, and that means Romney is done for.

CNN won’t project Florida – I don’t think anyone would dare project Florida, give what happened in 2000, but 86% of the vote is in, and Obama has the lead. The remaining 14% of the vote that remains to be counted are in Democratic stronghold territory, which means one thing – Obama gets 4 more years!

Also available in : Bulgarian

>>

, ,

The blame game

The mainstream media

Talk to a Ron Paul supporter for more than 5 minutes, and you will most certainly hear about the great conspiracy perpetrated against Ron Paul by the mainstream media.

It’s not just him though. In the republican primaries, Newt Gingrich went ballistic on the moderator for asking him to comment on his ex-wife’s allegations of infidelity. Gingrich seems to think that “much of the media” is spreading “vicious attacks”.

The other day I was watching Fox News at the gym – the talking head went on a rant about how the “mainstream media” is not reporting on the alleged Libya cover up.

What mainstream media is this guy talking about? Last time I checked, Fox had higher number of average daily viewers than MSNBC and CNN combined! 

I’ll go out on limb here and guess that you’ll never about the “mainstream media” from a politician who is getting good press. Somehow it’s only the people who are not taken seriously or the people who can’t get enough attention, who see a “mainstream” media conspiracy.

 

 

>>

,

Microsoft got cool? Apple is getting clobbered in the ad wars?

The new Iphone 5 has been out for some time now. The ads are back on the air – advertising the Iphone’s groundbreaking new features – the shape of the headphones! Who else but Apple would’ve thought that ear buds have to match the shape of the earlobe? And that you can use your thumb to  scroll your smart phone’s screen? Those are breathtaking innovations! I guess the 2% R&D budget is paying off…

For sure, the sales are through the roof, but Apple’s monopoly on cool is starting to slip. Samsung are clobbering them with their Samsung Galaxy S3 ads; the message is clear – the cool kids have moved on. Samsung is doing to Apple, what Apple did to Microsoft with the Mac vs PC commercials! And if Apple keeps pushing the same old products, it just might work!

Speaking of big old Microsoft – they got cool! I’m talking about the Surface table commercial that’s playing on TV. Snapping keyboards, lively, colorful, young people jumping around, dancing … The first few times I saw it, I didn’t even realize it’s a Microsoft commercial.

Whoever came up with that commercial has to be given an award. Compared to the weird and creepy Bill Gates & Seinfeld commercial, this one is an incredible improvement. I mean, who in their right mind will put Bill Gates in a commercial? Actually I know who – Apple.

I’m glad for them. Finally, 5 years after they introduced the multi-touch Surface table computer, Microsoft are doing something with the technology. The tablet look pretty good, but they have to get the cool factor working for them.

That’s extremely difficult. Microsoft is synonymous to uncool. They are not helping themselves either – copying everything from everyone, they’ve become target for ridicule  They even cloned Steve Jobs for the introduction of this very same Surface tablet!

I sure hope this new ad is a break with practice.

>>

, , , , ,

Google’s gtest not handling friend tests correctly

I was using Google’s gtest unit test framework today and I found an annoying problem – you can’t properly friend test methods that are in a different namespace than the class being tested.

Here is a typical situation:

namespace MyNamespace
{
   class MyClass
   {
      public:
         MyClass(int inData): data(inData){}
      private:
         int data;
   };
}//end namespace

namespace MyNamespace
{
namespace TEST
{
   class MyTestClassFixture
   {
      static const int fixtureData;
   }
   static const MyTestClassFixture::fixtureData = 5;

   TEST_F(MyTestClassFixture, TestEqual)
   {
      MyClass testClass(5);
      ASSERT_EQ(fixtureData, testClass.data);
   }
}
}

>>

, , , ,

Oracle C++ API non-sense

Parameterized statement throws invalid number SQLExceptions

I spent 2 hours trying to figure out this nonsense. Here is the problem:

Environment *env = Environment::createEnvironment();
Connection *conn = env->createConnection("username", "password",
connectionString);
string sqlQueryText = "UPDATE myTable SET field4 =:p4, field3 =:p3
WHERE field2 :=p2 AND field1 :=p1";
Statement* updateStatement = conn->createStatement(sqlQueryText);

updateStatement.setInt(1, field1Var);
updateStatement.setString(2, field2Var);
updateStatement.setInt(3, field3Var);
updateStatement.setString(4, field4Var);

updateStatement.executeUpdate(conn);

>>

, , ,

Guard you privacy

Using a password manager

In the first article of this series, I went on a pretty long rant about how people are not using secure passwords, how that could lead to their information being stolen, etc, etc.

In this one, I’ll go over what I’m using for my own online security. It’s all based on free software and services,which have been around for years and everyone probably knows about, but since I have not seen any comprehensive tutorials on how to do this, I’ll try to explain in details. Feel free to skip ahead if you think I’m wasting your time, you are not interested in the functionality I describe or whatever.

So enough beating around the bush…

>>

, , , , , , ,

Фонетичен Дворак за български

Фонетична подредба за български език

Когато реших че е крайно време да се науча да пиша по 10-то пръстната система, се разрових в Интернет и намерих системата Дворак. Според Интернет (а всички знаем че Интернет знае всичко и е безгрешен), Дворак се учи по-лесно, скороста на печатане е по-висока, по-ергономична е, намалят стреса на пръстите – абе с две думи само където не ти поднася закуска в леглото.

А може и да има такава опция, но да не съм я намерил още…

Научих Дворка и съм доволен. Има обаче един проблем. Никъде не можах да намеря фонетична Дворак подредба за Български език. Да седна да уча БДС клавиатурна подредба, не ми се занимава, а фонетичната която идва с Уиндоус следва АНСИ подредбата…

Този проблем ме мъчи от доста време, така че най-накрая реших да създам една фонетична Дворак подредба. Ето го резултата.

>>

,

Previous Posts