We should first initialize the microcontroller EUSART:

In the order of apparition, we first activate the 8 bits asynchronous mode that we then configure to run at 115200 bauds through the SPBRG register and BRGH bit. Reception is activated through the CREN bit and the interruptions are disabled. Finally, the TX and RX pins of the PIC are placed in input mode, as recommended by the manufacturer. Always pay attention to the TRISC5/4 lines to match the PIC you are using (these pins may change from one PIC model to the other).

Sending a byte is relatively easy:

After setting up the TXEN bit, the microcontroller will wait for the TXREG to be written to. Once the register has been changed, the byte is automatically sent. The microcontroller has finished sending the byte when the TXIF bit is set to 1. You should always wait the TXIF bit before sending a second byte.

Receiving data is a bit more complex:

Data is available for reading in the RCREG register when the RCIF bit is set to 1. If an error occurs, it is signalled by the OERR bit. Error clearance is reset by disabling and enabling reception again such as suggested by the manufacturer.

Finally, we may write a simple program to send the "Hello" string to the computer in loops:

The \r\n chars allows for a line feed and carriage return; it is the common way of having a new line in Windows. If you run Windows HyperTerminal with the correct settings, you should read a lot of "Hello" lines.

Once transmission of data is functional, we can check for reception by continually reading data and sending it back to the PC (echo mode):

The io_getc function has a blocking effect because it will loop forever until the RCIF flag is set to 1 to notify incoming data.

This is problematic when the main loop is in charge of other operations since everything will get to a halt until some data is received through the RS232 protocol. A good way to prevent this is to enable interruptions:

And the program becomes:

Do not forget to enable interruptions through the PEIE and GIE bits. This time, the main loop is empty but the microcontroller will jump to the ctrl function when an interruption (such as RCIF) is triggered.

Now that we are able to communicate through a HyperTerminal, it would be great to have our own software to communicate with the microcontroller. To make it easier to re-use, I have wrapped everything into a class object but it's not compulsory if you are not comfortable with Object-Oriented Programming.

Since Windows NT (this includes XP, Vista and 7 which are all derived from the NT kernel), any data exchange with drivers are programmed as file accesses. To communicate with the RS232 hardware, we actually have to communicate with its driver which will access the electronics of the computer for us. Just open a file named with the COM port number and write data to it!

The class stores a handle to the driver in a pointer m_hSerialPort. When the class is initialized, the handle is set to a value which means that it is invalid (which is right because we haven't opened any communication channel yet). When the class closes, we will call a close function to tell the driver we are done with it. Reading and writing will be done through the read and write functions.

Let's begin with these latter two:

In both cases, we are simply wrapping Windows function to read and write from/to files but we add a few lines to check that the handle is good and to cleanup things we a flush function. The flush function more or less have the same role has the OERR lines on the microcontroller side. I have defined it in the private part of the class:

Proper closing is without commentary:

All that is left is the open function. It's a bit longer but not that complex:

First we close any opened channels (in the case we would have called the open function twice or more without explicitly calling the close function - Windows do not like when we leave communication channels opened forever; this would be a resource leak). Then we open a communication channel to the specified port (such as "COM3"). We require access to read (GENERIC_READ) and write (GENERIC_WRITE) to the file and we also require it to exist (OPEN_EXISTING). If everything is successful, the handle will be a valid value. In any other cases (port not existing or already in use), the output will be INVALID_HANDLE_VALUE.

Once opened, we have to configure the port according to our settings. We begin by retrieving the actual port state through GetCommState and set our values (no parity bit, one stop bit, no flow control and the bauds speed). We then update the port state through SetCommState. If anything goes bad, we close the port and return a false value.

We may then imagine the following program which outputs anything read on the port (to be used with the "Hello" example):

This program opens the COM3 port at 115200 bauds and prints all incoming chars. The \0 thing is required by the printf function which will run crazy if we don't tell it where to stop printing. The data received is stored in a buffer of 128 bytes and the number of bytes read is returned by the read function. Always pay attention that there is no guarantee that the read function will always return 128 bytes! So always watch the actual number of bytes read. This is usually one of the first pitfalls of RS232 programming.

You may finally wonder why we do not call the close function explicitly. There is actually no need to do it in our small example because when the main function closes, the destructor of the SerialComm class will be automatically called. If you watch closely our class definition, you will see that the destructor explicitly calls the close function for us. This is a good example of what RIIA (a programming idiom which stands for "Resource Initialization Is Acquisition") can do for you!

We will create a communication standard containing several key information allowing data integrity and compactness of representation. These kinds of protocols are frequently met in IT and have various degrees of complexity. We will make our own protocol here with specifications adapted to our problem.

Any outputted data shall be sent as a series of bit representing the information as a packet. To be able to rebuild the data on the reception side, we will require two key features:

1/ When does the data packet begins. This is commonly implemented by always starting a packet with a known identified. In our previous example the identifier was "Temperature: ".

2/ How long is the packet. This information might be implicit such as when we know the packet size is always the same.

To assert data integrity once received, we also need a field that is a known mathematical function of the packet such that we can recompute the result on the received data and check if it matches the value sent along with the packet. We will refer to this as the checksum procedure.

I will now focus on packets that have always the same size, begin with an identifier and also take a checksum value. The previous temperature example would then translate in the form of structure as:

In our example, we are using a 32-bits identification field and a 16 bits checksum. We could also have used an 8-bits identifier:

Or even an 8-bits checksum:

In the former approach our header (identifier plus checksum) was taking 6 bytes but only 2 in the latter case. The former is more secure than the latter but also slower because it takes more data to send. The choice is always dependent on the application specification: are you reading temperature a hundred times per second or are you sending an ignition command to your rocket?

We will create a packet using an 8-bit identifier and an 8-bit checksum for header. The packet also contains timing information and two 16-bits values for analog reading:

The program then sends the value of the analog channels at regular intervals:

Every time the TMR1 register overflows (that is about every 10 ms with a 20 MHz quartz), it sets the flag g_ubSend to 1 which allows the main loop to read the analog channels and send the result to the computer. Even if it looks complicated, all the main function does is waiting for g_ubSend and fill the packet with data.

The write function is only successive calls to io_putc:

The checksum function is a bit more interesting. I have derived it from the TCP/IPv4 norm of the RFC793. It returns the 2-complement of the sum of the 2-complements of every byte in the packet:

Don't forget to put the checksum field to 0 before calling the checksum function or the result will not be reproducible on the PC.

On the computer side, we now have to read the information. For more clarity, I have packed everything in a Protocol class:

The class allocates a buffer memory that is required for the reception of data (it is of 1 kb by default but you can tune it depending on your application). Again, the destructor of the class will manage everything and free the memory allocated.

The checksum function is the same as on the microcontroller side:

And the dirty things happen now! When a data is read, it should be sent to the onRead function of the Protocol class which will add it to the buffer memory. After each addition, the processBuffer function will be called to check if there is any data to be processed:

The processBuffer function first checks if there are enough bytes in the buffer to have a complete packet. If not, it returns and waits for new data addition.

If there is enough data to potentially make a packet, it will read the first byte(s) to check the identification field. If the identification field does not match the beginning of a packet, we may have ended up in the middle of a packet or with erroneous data. We then skip the first byte and try again the whole procedure until we have caught the beginning of a packet.

Then, we store the value of the packet checksum and compute the checksum again (don't forget to set the field to 0 to make it reproducible). If the checksum does not match, it means that something wrong happened with the packet and so we can discard the whole packet and wait for new data. There is always a small risk that the identification field do not catch the actual start of a packet (there is a 1/255 odds with 8-bits identification) but only some data inside a packet; but in that case, the checksum will reveal that out too. Still, there is a 1/65535 odds that we have incorrect data with valid checksum and valid identification... If you are not happy with it, increase the identification and checksum fields bit sizes.

The swapBuffer function is easy to implement:

We may then create a program that prints the two analog values sent by the PIC:

As said previously, the microcontroller side is just an adaptation of what we have done on the computer except that we have to get things working with much less RAM:

On the computer side, there are much more modifications to do. First, we will have to activate the Overlapped I/O mode and create a separate thread which will read the data received.

We then begin by adding a handle to the thread to our SerialComm class:

And we shouldn't forget to stop the thread when the close function is called:

The function WaitForSingleObject allows waiting for 1 second that the thread exits nicely before killing it. The function CancelIoEx cancels all pending operations on the driver. The role of m_bForceQuit will be highlighted later.

We then have to modify the read and write function:

The open function will have to be modified to create a thread and ask for the Overlapped I/O mode (FILE_FLAG_OVERLAPPED):

The rest of the function remains unchanged and was not copied here. The thread created is sent to the entry point asyncRead with a pointer to the current SerialComm object which allows calling the asyncRead function inside the object. It might look weird but it's the only way for Windows thread to have entry function inside a class:

With the latter function:

It first creates a 128 bytes buffer and some structure required for Overlapped I/O. It then creates an event which will be used to trigger data reception. We then call ReadFile which no longer blocks and so we have to use the event to check if data is available. We wait data for only 100 ms so we can use the m_bForceQuit flag to force the thread to exit. Most electronic enthusiast will probably be starring at this code with big eyes but don't worry if you don't understand it completely; it's the result of several optimizations I wrote.

To process data we then call the pure virtual function onRead which has to be defined in children of SerialComm:

This is not of the best practice in terms of design patterns but it's the easiest way to make it working. I will show you a more advanced implementation later.

Finally, the main function of our program becomes:

Without forgetting the Protocol class:

The program simply opens the COM3 port in asynchronous mode, send the value 0, wait 1 second and send the value 4095. The program then waits for 5 more seconds and closes the stream. Remember that the program has no longer blocking features and if you don't tell it when to stops it will close immediately. For example, the following code will return immediately:

So far, we have taken for granted that we knew the port number to use. This is often not the case, especially when there are several serial devices plugged on the same computer. It may therefore be interesting to be able to list all the port on the local computer.

Hopefully, this list is readily available in the Windows registries and can be accessed that way:

The enumerateSerialPorts function opens the registry key HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\SERIALCOMM and lists its content. Every entry is then checked with isSerialPortAvailable (to check for already opened ports) and the positive results sent to a callback function pfnEnumerateSerial. It is important to restrict the request to read only mode (KEY_READ) as a generic access (KEY_ALL_ACCESS for example) might be rejected by Windows UAC if the user do not have the rights to access the key in other modes.

Since the function was defined static, it can be accessed that way:

When working with asynchronous I/O, it is possible to use the ReadFileEx function which allows using a callback function once data are read. It might then be tempting to use that functionality rather than the thread we have been using.

However, the callback function can be called only if the process is placed in an alert state through a specific function such as SleepEx. Since that mechanism is not fully under our control and make the communication code dependent on the application, we should restrict from using this functionality and rely on the thread version developed here.

It might also be tempting to use the computer clock to tell the exact moment a data has been received rather than adding a field to the packet sent such as we did in a previous example. This is unfortunately not possible for two major reasons:

1/ Windows do not guarantee that data are presented at the exact time they are received. Sometimes, the system will idle after incoming data.

2/ We usually receive data as burst containing several packets because of the buffer memory. Still, even by reducing the buffer size we can fall in the previous issue.

Earlier I have mentioned that the present design was not perfect with the Protocol class inheriting from the SerialComm class. This is because a protocol and a communication channel are two different things. By acknowledging this, we can design a more robust implementation which allows several protocols to be used on the same communication channel. I am only giving here some pseudo-code and I will let you write your own final version of it:

Every protocol object has one and only one SerialComm object as source but a SerialComm object may have a list of several protocols attached to it:

By changing the SerialComm onRead function to a dispatch function:

And having a formal definition of proccessBuffer for every child of the Protocol interface:

One could then only write:

Following the Microsoft Developper Network, to access a port above " COM9 " (such as " COM10 ") you should prefix the name with "\\.\". That way, COM10 should then be accessed as "\\.\COM10". You may, of course, do the same for other ports such as "\\.\COM3".

#pragma pack commands often pop ups in my code and I did not give any explanation about that. It is a command sent to the compiler to tell how data should be packed. By using #pragma pack(1) we tell the compiler to align the data at the byte level. By default, Visual Studio packs everything at the dword level while the PICCLITE does it at the byte level. If you don't pay attention to this, the packet structure of the computer and the packet structure of the PIC will not be the same and communication will fail.

So don't forget to write:

So far our program performs well as long as the data keep coming. On the other hand, if we write a PIC program that sends only a few data spaced by long intervals, nothing arrives on the computer until the buffer memory gets full. The work-around for this is to tell the driver we would like to receive data not only when the buffer is full but also after some time-out delay.

I have chosen here to set the time out period to 100 ms. All we have to do is to modify the open function:

Let us imagine a microcontroller which sends 10-bits analog data to the computer at regular intervals that we would like to use as a fast data logger. With our system, we know that a packet should contains an 8-bits identification field, an 8-bits checksum, a 10-bits data field and some clock field to check for missing packets. To make the system as fast as possible, we will use a 6-bits clock field making the whole packet 4 bytes long.

Using a transfer speed of 115200 bauds, we know that we can send about 2600 packets per second. We have to add to this the time required to actually read the analog data using the microcontroller ADC. Let's say the microcontroller needs a 0.1 ms to read the data giving an overall execution speed of about 2 kHz. We may increase it by a little using the PIC interruptions for the ADC and send the data over the TX pin while we are reading the next analog value. This is possible because most of the time the microcontroller is idling. At best, we could increase the speed up to 2600 Hz which is the maximum packet density we may achieve.

But we can still make better by using buffered sending. If we pay attention to our packet structure, 50% of the packet contains only identification data. If we could send 2 analog values at a time, this would now be only 33% of the packet... The more data we send in one packet, the better we are using the bandwidth for actual data. Let's say, we can send one hundred analog values at a time. Since we know that each data is spaced by a known time interval, we only need one clock field per packet. So, in total, we have 100 * 10 bits (data) + 16 bits (header) + 8 bits (clock) which is 128 bytes. Using this 128 bytes packet, we could increase the average transfer speed to about 8.2 kHz! This is far better than the previous 2.6 kHz...

Still, we have a problem: we are getting burst of 100 data followed by a long silence and then again a burst of data. This is because we have to fill the buffer with analog values and send it only when the buffer is full.

There is, again, a trick which is to use two buffers: a front buffer and a back buffer. The main loop sends the back buffer over the communication channel while, at the same time (using the interruption mechanism), the PIC writes the front buffer with the analog data. When the front buffer is full, the PIC just swap the front and back buffer such that the system never stops. There is only one condition for this to work: the back buffer should be completely sent before the front buffer gets full. Put differently, the acquisition speed must be slightly lower than the sending speed. If we are sending data at 8.2 kHz (measured), we should set the acquisition timer to 8 kHz and no more.

The number of data we can pack in one packet is limited by the available RAM of the PIC. But don't forget that you need to store 2 buffers and not one so a PIC with 256 bytes of RAM can only store a maximum of two 128 buffers.