Search This Blog


Monday, April 27, 2015

PHP IPC with Daemon Service using Message Queues, Shared Memory and Semaphores - PHP Classes blog - PHP Classes

PHP IPC with Daemon Service using Message Queues, Shared Memory and Semaphores - PHP Classes blog - PHP Classes


In a previous article we learned about Creating a PHP Daemon Service. Now we are going to learn how to use methods to perform IPC - Inter-Process Communication - to communicate with daemon processes.

Message Queues

In the world of UNIX, there is an incredible variety of ways to send a message or a command to a daemon script and vice versa. But first I want to talk only about message queues - "System V IPC Messages Queues".
A long time ago I learned that a queue can be either in the System V IPC implementation, or in the POSIX implementation. I want to comment only about the System V implementation, as I know it better.
Lets get started. At the "normal" operating system level, queues are stored in memory. Queue data structures are available to all system programs. Just as in the file system, it is possible to configure queues access rights and message size. Usually a queue message size is small, less than 8 KB.
This introductory part is over. Lets move on to the practice with same example scripts.
// Convert a path name and a project identifier to a System V IPC key $key = ftok(__FILE__, 'A'); // 555 for example  // Creating a message queue with a key, we need to use an integer value. $queue = msg_get_queue($key);  // Send a message. Note that all required fields are already filled, // but sometimes you want to serialize an object and put on a message or a lock. // Note that we specify a different type. Type - is a certain group in the queue. msg_send($queue, 1, 'message, type 1'); msg_send($queue, 2, 'message, type 2'); msg_send($queue, 3, 'message, type 3'); msg_send($queue, 1, 'message, type 1');  echo "send 4 messages\n";
$key = ftok('queue-send.php', 'A'); // 555 for example
$queue = msg_get_queue($key);  // Loop through all types of messages. for ($i = 1; $i <= 3; $i++) {     echo "type: {$i}\n";  // Loop through all, read messages are removed from the queue. // Here we find a constant MSG_IPC_NOWAIT, without it all will hang forever.     while ( msg_receive($queue, $i, $msgtype, 4096, $message, false, MSG_IPC_NOWAIT) ) {         echo "type: {$i}, msgtype: {$msgtype}, message: {$message}\n";     } }
Lets run on the first stage of the file queue-send.php, and then queue-receive.php.
u% php queue-send.php send 4 messages u% php queue-receive.php type: 1 type: 1, msgtype: 1, message: s:15:"message, type 1"; type: 1, msgtype: 1, message: s:15:"message, type 1"; type: 2 type: 2, msgtype: 2, message: s:15:"message, type 2"; type: 3 type: 3, msgtype: 3, message: s:15:"message, type 3";
You may notice that the messages have been grouped. The first group gathered 2 messages of the first type, and then the remaining messages.
If we would have indicated to receive messages of type 0, you would get all messages, regardless of the type.
while (msg_receive($queue, $i, $msgtype, 4096, $message, false, MSG_IPC_NOWAIT)) { // ... 
Here it is worth noting another feature of the queues: if we do not use the constant MSG_IPC_NOWAIT in the script and run the script queue-receive.php from a terminal, and then run periodically the file queue-send.php, we see how a daemon can effectively use this to wait jobs.
$key = ftok('queue-send.php', 'A'); // 555 for example
$queue = msg_get_queue($key);  // Loop through all types of messages. // Loop through all, read messages are removed from the queue. while ( msg_receive($queue, 0, $msgtype, 4096, $message) ) {     echo "msgtype: {$msgtype}, message: {$message}\n"; } 
Actually that is the most interesting information of all I have said. There are also functions to get statistics, disposal and checking for the existence of queues.

Lets now try to write a daemon listening to a queue:

// Fork process $pid = pcntl_fork(); $key = ftok('queue-send.php', 'A'); $queue = msg_get_queue($key);  if ($pid == -1) {     exit; } elseif ($pid) {     exit; } else {     while ( msg_receive($queue, 0, $msgtype, 4096, $message) ) {         echo "msgtype: {$msgtype}, message: {$message}\n";     } }  // Disengaged from the terminal posix_setsid();

Shared Memory

We have learned to work with queues, with which you can send small system messages. But then we may certainly be faced with the task of transmitting large amounts of data. My favorite type of system, System V, has solved the problem of rapid transmission and preservation of large data in memory using a mechanism called Shared Memory.
In short, the data in the Shared Memory lives until the system is rebooted. Since the data is in memory, it works much faster than if it was stored in a database somewhere in a file, or, God forgive me on a network share.
Lets try to write a simple example of data storage.
// This is the correct and recommended way to obtain a unique identifier. // Based on this approach, the system uses the inode table of the file system // and for greater uniqueness converts this number based on the second parameter. // The second parameter always goes one letter $id = ftok(__FILE__, 'A');  // Create or open the memory block // Here you can specify additional parameters, in particular the size of the block // or access rights for other users to access this memory block.  // We can simply specify the id instead of any integer value $shmId = shm_attach($id);  // As we have shared variables (any integer value) $var = 1;  // Check if we have the requested variables. if (shm_has_var($shmId, $var)) {     // If so, read the data     $data = (array) shm_get_var($shmId, $var); } else {     // If the data was not there.     $data = array(); }  // Save the in the resulting array value of this file. $data[time()] = file_get_contents(__FILE__);  // And writes the array in memory, specify where to save the variable. shm_put_var($shmId, $var, $data);  // Easy? 
Run this script several times to save the value in memory. Now lets write a script only to read from the memory.
// Read data from memory. $id = ftok(__DIR__ . '/shared-memory-write-base.php', 'A'); $shmId = shm_attach($id); $var = 1;   // Check if we have the requested variables. if (shm_has_var($shmId, $var)) {     $data = (array) shm_get_var($shmId, $var); } else {     $data = array(); }  // Iterate received and save them to files. foreach ($data as $key => $value) {     // A simple example, create a file from the data that we have saved.     $path = "/tmp/$key.php";     file_put_contents($path, $value);      echo $path . PHP_EOL; } 


So, in general terms, it should be clear for you by now how to work with shared memory. The only problems left to figure out are about a couple of nuances, such as: "What to do if two processes want to record one block of memory?" Or "How to store binary files of any size?".
To prevent simultaneous accesses we will use semaphores. Semaphores allow us to flag that we want to have exclusive access to some resource, like for instance a shared memory block. While that happens other processes will wait for their turn on semaphore.
In this code it explained clearly:
// Let's try to save a binary file, the size of a couple of megabytes. // This script does the following: // If there is input, it reads it, otherwise it writes data into memory // In this case, when writing to the memory we put a sign lock - semaphore  // Everything is as usual, read the previous comments 
$id = ftok(__FILE__, 'A');  // Obtain a resource semaphore - lock feature. There is nothing wrong if we // use the same id that is used to obtain a resource shared memory 
$semId = sem_get($id);  // Put a lock. There's a caveat. If another process will encounter this lock, // it will wait until the lock is removed 
sem_acquire($semId);  // Specify your like picture 
$data = file_get_contents(__DIR__.'/06050396.JPG', FILE_BINARY);  // These can be large, so precaution is necessary to allocate such a way that would be enough 
$shmId = shm_attach($id, strlen($data)+4096); $var = 1;  
if (shm_has_var($shmId, $var)) {     // Obtain data from the memory           $data = shm_get_var($shmId, $var);      // Save our file somewhere     $filename = '/tmp/' . time();     file_put_contents($filename, $data, FILE_BINARY);      // Remove the memory block that started it all over again.     shm_remove($shmId); } else {     shm_put_var($shmId, $var, $data);  }  // Releases the lock. 
Now you can use the md5sum command line utility to compare two files, the original and the saved file. Or, you can open the file in image editor or whatever prefer to compare the images.

With this we are done with shared memory and semaphores. As your homework I want to ask you to write code that a demon will use semaphores to access shared memory.


Exchanging data between the daemons is very simple. This article described two options for data exchange: message queues and shared memory.

Post a comment here if you have questions or comments about how to exchange data with daemon services in PHP.