Producer Consumer (C)

From LiteratePrograms

Jump to: navigation, search

This short LIME program provides an example of concurrent programming using data-flow paradigms.

We illustrate this using a simple data-flow graph, containing two components:

  1. producer, that generates consecutive integers starting from 0, in pairs
  2. consumer, that verifies that consecutive integers are received, one by one

The dependencies between these components (implemented as C modules) are encoded in an XML using GXF schema. Note that you can visualize and edit this GXF specification graphically, using the LIMEclipse plugin [1].



So, lets start with the producer, that has to be specified in a separate C module (note that __LIME__ is defined when the module is compiled using slimer, and not otherwise):

#ifdef __LIME__ 
/* for slimer compilation */
#include LIME
/* for regular compilation */
#include "LIJM.h"
actor producer(OutPort int out[2])
    static int state = 0;
    out [0] = state++;
    out [1] = state++;

That was rather easy. We use #include LIME to indicate that this C module is also a LIME module. It also serves as a hook onto which the LIME compiler hangs the definitions of various LIME "keywords" such as actor and OutPort. These are actually just syntactic sugar (actor=='void', OutPort==' '), provided by LIME to improve readability.

For the rest, its rather obvious. Note that the rate with which the producer generates integers is explicit in the source-code as an array size specifier for the out port. The data that's available on this port is directly accessible via the function argument as a pointer/array of 2 integers.

Also note that producer uses a local static variable, which precludes multiple instantiation of this module. Although its possible to improve that, it would unnecessarily complicate this simple example too much (basically you need to define state as a structure, require it in all related modules and declare each module's state as a port of the related module).

That's why I don't tag this example yet as Parallel or Multicore, but that shall be solved soon enough:-)


The consumer module doesn't contain many surprises, the only new "keyword" is InPort (defined as InPort=='const') which adds the const modifier to the in port. This prevents stray code to modify the supposedly immutable input data.

Note that a module can also include "system" headers for things such as assert().

#ifdef __LIME__ 
/* for slimer compilation */
#include LIME
/* for regular compilation */
#include "LIJM.h"
#include <assert.h>
actor consumer(InPort int in[1])
    static int state = 0;
    assert(in[0] == state++);


Finally, we need to specify that producer:out connects to consumer:in. While we could have used a notation ala Graphviz (producer:out -> consumer:in), currently, LIME mandates an XML schema called GXF that vaguely resembles GXL:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE stream PUBLIC '-//LIME/DTD stream 1.0 Transitional//EN' ''>
<stream xmlns:xi='' xmlns:xlink='' xlink:type='extended'>
<edge type="fifo"><from-node id='producer' port-id="out"/>
                  <to-node id='consumer' port-id="in"/></edge>

That's it! Now, provided that you have installed LIME and some appropriate back-end for your system (LIME-Pthread, LIME-CUDA etc.), you can just do slimer -o test producer.c consumer.c -- -- test.graph. This should result in a ready-to-run executable called test.

LIJM (LIME glue)

Because its not trivial yet to install LIME compiler, here we provide a glue header ("lijm" means "glue" in Dutch) that can be used to compile LIME C modules even without slimer. Although its possible to let the C modules just use #include LIME and let LIME symbol be defined via the command-lime to the compiler, i.e., cc -DLIME="LIJM.h", we don't do this here, since that would result in error-messages from the back-end preprocessor installed on this site.

The examples above use either #include LIME or #include "LIJM.h" guarded by an internal LIME-defined pre-processor flag), which could have been avoided if this site allowed us to specify special command-line defined pre-processor flags.

Of course then you'll need to provide the wrappers that create real tasks out of actor functions above, startup and cleanup code for these and the build environment in which actual executables can be produced.

#ifndef LIJM_defined
#define LIJM_defined
#define Typedef(tag,type) typedef tag type type##_t
typedef void actor;
typedef actor (*actor_t)(int);
typedef void subactor;
typedef subactor (*subactor_t)(int);
typedef void (*controller)();
typedef subactor_t (*controller_t)();
typedef struct {
 	    void (*fun)();
	} reactor;
typedef reactor (*reactor_t)();
typedef unsigned int proactor;
typedef proactor (*proactor_t)();
#define _static
#define _volatile
#define _const
#define _restrict
#define InPort const
#define OutPort
#define InOutPort
#define SharedOutPort	volatile OutPort
#define SharedInPort	volatile InPort
/* lowlevel macros */
#define numports _const _static _restrict
#define NumPorts(n) numports n
#define gen_rate _static
#define opt_rate /*empty*/
#define _Rate(r) gen_rate r
#define _OptRate(r) opt_rate r
#define datarate _restrict _static 
#define doptrate _restrict
#define DataRate(r) datarate r
#define OptDataRate(r) doptrate r
#define DataVar DataRate(1)
#define staterate _volatile _static 
#define soptrate _volatile
#define StateRate(r)  staterate r
#define OptStateRate(r) soptrate r
/* generic macros */
#define Rate(r) 	DataRate(r)
#define OptRate(r) 	OptDataRate(r)
#define Required	Rate(1)
#define Optional 	OptRate(1)
#define StateVar	StateRate(1)
#define State		StateVar
#define OptState 	OptStateRate(1)
#define OptData 	OptDataRate(1)
struct edge {
    char* type;
    struct endpoint {
	void (*node)(int);
	char* port;
    } from,to;
/* Hide Lime pragmas to the tail toolchain */
#define Lime(x)
/* Common defines */
#define lengthof(type)  (sizeof type/sizeof type[0])
/* Associative operations for collectives */
#define max(a,b) ( (a) >= (b) ? (a) : (b) )
#define min(a,b) ( (a) <= (b) ? (a) : (b) )
#define equ(a,b) ( (a) == (b) )
#define sum(a,b) ( (a) + (b) )
#define prod(a,b) ( (a) * (b) )
/* g is the granule size */
#define roundup(s,g)	((s)/(g)+!!((s) % (g)))
#define round_up(t,g)	roundup(sizeof(t),g)
/* l is log2 of the granule size */
#define loground(s,l)	(1+((((s)-1) & ~((1<<(l))-1))>>(l)))
#define log_round(t,l)	loground(sizeof(t),l)
/* LIJM_defined */
Download code