Parallel processing, daemons, forks, threads

Forks, threads, and daemons

  • Parallel processing is used on all modern computers without the knowledge of the user today. Even modern smartphones have time-shared multitasking and in fact symmetric multi-processing (SMP support) with their multi-core processors. Yet the average application software designer and developer still lives in a strictly sequential world. We use parallel processing in business applications we design extensively, to get faster response times and manage workloads better.

  • Daemons Having grown up in the world of Unix systems, we look upon daemons with a friendly eye. We have often used long-running background processes (i.e. daemons) to take up part of the workload when the front-end systems would have become overloaded. For instance, if a user interface is expected to generate a complex ad hoc report from a database, a complex synchronous SQL query will make the user impatient. In such situations, we create a background process which runs the queries and stores the results in a text file or HTML page. The UI code merely queues a request message for this background process, asking for the report the user needs. After this, the UI displays a message to the user informing him that his report has been queued for generation. The user may return to the same page at any future time to view the report if it has already been generated. Since the report-generating daemon fires only one complex SQL query at a time, the load on the database is kept within limits, improving overall system response for interactive transactions and other rapid queries.

    A background daemon is also extremely useful is for maintaining a shared repository of information. For instance, if there is a need to maintain a persistent set of key-value pairs, then querying and updating a SQL table each time may be much slower than using UDP packets to communicate with a background process which holds this data in memory. In some sense, this is what the memcached daemon provides. Such daemons become even more attractive if the information structure is variable and an update may contain one or a dozen attributes in one packet. This sort of information may also be maintained in a shared Berkeley DB file, but mutex locks on the file may tend to slow things down -- a dedicated daemon does not need to lock its own data store.

  • Forking The fork() system call in Unix allows a process to fire a child process. We use child process creation quite regularly in business applications. One situation where this helps is when the user triggers an operation through the UI which will take a long time to complete. Instead of completing the operation synchronously and making the user wait, a child process is forked and the parent returns immediately saying "Your operation will be completed right away." The child process completes the job in the next few seconds or minutes.

    The forking of child processes may appear to be an alternative to maintaining one's own pet daemon, as was described above. However, the two are not really alternatives. A daemon is useful when exactly one instance of a processing engine is desirable. Child processes are created at will when there is no such need to maintain a single background process, and multiple child processes may be permitted to run concurrently.

  • Threads Threads are exciting solutions to problems when two-way information sharing is needed between the requester and the back-end process. The Unix process model makes shared-memory sharing and semaphores a bit difficult for the programmer to use. In the absence of shared-memory segments, a Unix parent process can give its child a snapshot of its entire process state, but there is no way for the child to update the parent's memory state. Threads win hands down here.

    Java has made thread programming easy. We have created threads from within servlets on many occasions, allowing the parent thread to return a response to the user while the child thread continues processing of data for longer-running requests. These appear to be no-brainers to us and we are shocked at how rarely a typical business application designer uses these mechanisms.

    We have used these techniques to make ordinary servlet-based Web applications more "intelligent". In one case, we created a thread in the Tomcat JVM at start-up, which would keep monitoring system health. Other threads were created and destroyed by the Tomcat servlet container to serve HTTP requests as is normally done. The health-monitor thread would update an in-memory singleton Java object whenever it found cause for concern. All servlets would check this object before they began their processing. If the health-monitor status object indicated any problems, all servlets would inform their users that the system was experiencing health problems, "please come back later."

    We have also created special batch-processing applications in Java which make full use of modern SMP hardware. Such an application creates one set of worker threads to perform data processing operations and another set of threads to fetch data. A queue of jobs is used as a shared data structure (it could be as simple as a shared ArrayList in a critical region) to pass work from the data-fetch threads to the worker threads. The data-fetch threads may block on I/O, but this will not slow down the worker threads. This design, with careful tuning of the number of worker threads, can maximise throughput for batch processing applications. Thread support in Java makes such coding easy.