data to remember

For 60fps video, it takes 16.668333ms for each frame. 720×480, 28 ((1 / 16.668333 ) * 480) = 28 ) lines are transferred per ms. Practically, the lines are more than this number, as 16.668333 counts all the time, including the time for sync signals. What I saw is around 16ms (16.1, or 16.2, might be 16, or even a few us shorter than 16ms ).

programming tips: pre-filled pattern

Pre-fill a buffer with a pattern, and check how many lines have been changed, after a certain operation (e.g. transferring some image data from external interface into memory)
// the buffer is UYVY format, with each two bytes representing one pixel. The code writes 00000000 to the middle of each line

uint32_t *write_pos = (uint32_t *)(buf->pointer) + buf->stride /8;

int nlines = 0;

for(; nlines < buf->height; write_pos += buf->stride /4, nlines++) {

*write_pos = 0;


// Read each line from the top. If we only concern the bottom part, reset the counter “lines_untouched“, the value is changed. If we want to count every “hole”, no reset is needed.

uint32_t *read_pos = (uint32_t *)(buf->pointer) + buf->stride /8;

int nlines = 0;

int lines_untouched = 0;

for(; nlines < buf->height; read_pos += buf->stride /4, nlines++) {

if(*read_pos == 0) {



else {

lines_untouched = 0;



IPC: Message fundamentals

Channel: Servers receive on channels.
Connection: Clients should connect/attach to the channel first, then send to the channel using the connection.
Channel creation
int ChannelCreate(unsigned flags)
_NTO_CHF_PRIVATE    only be used inside a process
_NTO_CHF_FIXED_PRIORITY  receiving threads won’t change priorities to those of the sending threads.
_NTO_CHF_THREAD_DEATH  ask to deliver a pulse if a thread is dead.
_NTO_CHF_DISCONNECT  Deliver a pulse when all connections from a process are detached.
_NTO_CHF_COID_DISCONNECT Deliver a pulse to all connections if the channel is destroyed.
Connection attach/detach
Int coid = ConnectAttach(node_id, pid, chid, index | _NTO_SIDE_CHANNEL, flags)
_NTO_SIDE_CHANNEL should always be used, to make sure the returned coid is greater than any valid file descriptor.
When _NTO_SIDE_CHANNEL is used, index is ignored.
Int ConnectDetach(coid)
How does the client find the server?
Before MsgSend(coid, &msg, ..), a connection attach is required, coid = ConnectAttach(nd, pid, chid…).
  • If they are in same process, it is easy. Sharing “chid” across the process should work.
  •  If they are not (common case), below is how…
If server is a Resource manager
  • The server: resmgr_attach(..,”/dev/mymgr/”,…);
  • The client: fd = open(“/net/nodenme/dev/mymgr”, ..);
    • Write(fd,..), read(fd,..), MsgSend(fd,…)
    • Close(fd);
    • note: fds are a particular type of coid.
If server is just A simple MsgReceive() loop,
  • server: name_attach(NULL, “myname”, 0);
  • client: coid = name_open(“myname”, 0);
    • MsgSend(coid, &msg,….);
    • name_close(coid);

Cause of capture noise (1): power supply

My set up is shown below.
Two boards, which are considered identical, except that: board A is coming with camera_A hard-wired, I reworked board B to have a RCA connector.
DVD player is external powered, camera can use the external power supply (camera_B), or battery inside(camera_C), or get the power from the board (camera_A).
There is no noise pattern seen with camera_A + board_A. However, when connecting board_B to
  •  DVD player, there are noise patterns captured. The fact is,
    • The DVD player works well with most boards, even the ones with the same decoder as board B.
    • Switch to another DVD player, the noise pattern is still there.
    • I did have some experience that increasing drive strength of the decoder avoided the noise patterns, on an old board. However, I have no idea how to avoid it on this board.
  • Camera B, noise pattern is seen, and even worse than DVD player: the pattern keeps moving vertically, and the capture module lost sync with the decoder frequently (when the noise pattern moves to the vsync interval, I guess).
I need eliminate the noise patterns of camera, to have some fair comparisons between camera A and camera B.
What I did:
  1. As the cable between cameras and decoder was re-worked by myself, I thought I brought some noises in while just twisting the wires together.
    • However, even I soldered the wires tightly, and made sure they were not exposed in the air (someone told me the wire would act as antenna if exposed in the air, and receive some noise signals, consequently), the noise pattern is still there.
    • Moreover, Camera_C doesn’t have noise pattern, even it is connected via the same wires.
  2. As camera_C doesn’t have this issue, we started looking into the power supply. A few options available:
    • Use the power from the board (not done,due to insufficient knowledge of the wire definitions on that board).
    • Use an alternate power supply, what we tried:
      • A 7.5V power supply with the same DC connector type (the noise is even worse, 3,4 stripes).
      • Rework the power cable to get power from laptop, via USB (failed, as USB can provide 5V maximum, but this camera requires a higher voltage (12V)).
      • Find a stable power supply.
        • Luckily, get one stable power supply. The connectors of this power supply is MOLEX connector. Therefore, cut and twist the wires of MOLEX connector with the camera power cable… and finally the noise pattern disappeared!!!!
  • DC connectors: Coaxial connector; Molex connector (4 pins, used for disk driver connectors).
  • USB connector pinout: Red is for VCC, black is for ground. White and green are for data.
  • Molex connector Pinout: yellow wire: 12V; red wire: 5V, black wires: ground.
Update: there is one more cause which is worth to check: clock jitter resulted by wrong circuit design on the receiver side.
the two reasons above (low drive strength, or power supply) doesn’t introduce noise pattern if it is in Free run mode. If we do see noise even in free-run mode, check the circuit design.
See the image below. Removing R12102 and IC12102, and install R12100, will avoid the green line noise.
Screen shot 2015-07-10 at 11.02.35 AM
the IC is a bilateral switch, which makes the circuit tolerant of slower input rise and fall times when OE is low.

Thread synchronization: Condvar

A condvar (condition variable) is used together with a mutex lock. It blocks a thread within a critical section until some condition is satisfied.
Initialization and destroy
  •  pthread_condattr_init(pthread_condattr_t *attr;);
  • pthread_cond_init(pthread_cond_t *condv, pthread_condattr_t *attr);
  • pthread_cond_destroy(pthread_cond_t *condv);
Condvar attributes
    • pthread_condattr_setshared(&attr, int shared);
    • pthread_condattr_getshared(&attr, &shared);
  •  A condvar can use different clocks: CLOCK_REALTIME, CLOCK_SOFTTIME, CLOCK_MONOTONIC (not affected by changes to the system time). It will be used to for pthread_cond_timewait(). The default value refers to the system clock??
  • pthread_condattr_setclock(&attr,clock_id)
  • pthread_condattr_getclock(&attr, *clock_id);
  • pthread_cond_wait(&condv, &mutex);
    • this function blocks the calling thread on the conditional variable (condv), and unlocks the associated mutex (mutex).
    • the thread is blocked until
      • another thread calls pthread_cond_signal() or pthread_cond_boradcast(), or
      • a signal is delivered to the thread, or
      • the thread is canceled.
        • In all cases, the thread reacquires the mutex before being unblocked. It means, even another thread calls _signal(), or _broadcast(), this thread might still be blocked, until the other thread calls pthread_mutex_unlock() to release the mutex.
  • pthread_cond_timedwait(&condv, &mutex, const struct timespec *abstime);
    •  returns
      • EOK, if success, or was interrupted by a signal.
      • ETIMEOUT if timeout.
      • EPERM: the thread doesn’t own the mutex.
      • If the thread is cancelled, the cleanup handler is executed, after the thread reacquires the mutex.
  • pthread_cond_signal(&cond): unblock the highest priority thread.
  • pthread_cond_broadcast(&cond): unblock all the threads.
Useful tips
  • A thread must lock mutex first,  then wait for condv (On return from the function, the mutex is again locked and owned by the called thread).
  • Don’t use a recursive mutex with condition variables.
  • Make sure that the thread’s cleanup handlers unlock the mutex.
  • use while loop while waiting for a condvar. reasons:
    • POSIX cannot guarantee no false wakeups (e.g. Multiprocessor systems);
    • we need test to ensure the condition matches our criteria.
  • pthread_cond_timedwait() uses struct timespec to specify an absolute timeout value.
    • Use clock_gettime(clock_id, &ts) to get the system time
      • Don’t use the kernel call ClockTime_r(clock_id, const &new, &old)
  • More on get time funtions
    • ClockTime_rreturns the time in nano seconds, while clock_gettime() returns struct timespec.
    • two useful functions:
      • void nsec2timespec(&tp, nsec);
      • uint64_t timespec2nsec(&tp);
 Example(30ms timeout)
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
ts.tv_nsec += 30 * 1000000ULL;
// form 1
while (pthread_cond_timedwait(&condv, &mutex, &ts) == EOK) {
// get the condv, or interrupted by a signal
   If(condition is true) {
// an error returned by pthread_cond_timedwait(), possibly ETIMEOUT.
// form 2
While(contition is false) {
   If(pthread_cond_timedwait(&condv, &mutex, &ts) != EOK) {

Artifacts(1): alias and chromatic aberration

SChromatic aberration/distortion
Chromatic aberration occurs along boundaries that separate dark and bright parts of the image.
  • Reason: the lens fails to focus all colors to the same convergence point.
  • avoid: It can be reduced by inrceasing the focal length of the lens.
Spatial alias
 an effect that causes different signals to become indistinguishable, which can occur in sampling or reconstruction process. If the image data is not properly processed, the reconstructed image will differ from the original image. Usually introduced when representing a high-resolution image at a lower resolution.
Spatial anti-aliasing
Remove higher signal components which can’t be handled properly before sampling at a lower resolution.
Anti aliasing makes polygon edges not jaggy but smoothed out. The level of anti-aliasing determines how smooth polygon edges are.


Message and pulse are QNX native IPC, no POSIX/UNIX.
Usually, message passing is synchronous:  after MsgSend(), the client blocks(send-blocked, or reply-blocked) until the server gets around to replying.
If you want asynchronous messaging,
  1. if you need to transfer data
    1. Have the server dedicate a thread to receive and immediately reply messages, minimizing the blocking interval.
    2. Have the client send the data via a thrad
    3. Use POSIX message queue
  2. If you don’t need to transfer data
    1. Posix signal
    2. pulse
Pulse is asynchronous. Pulse makes “a non-blocking send” possible, which works for the scenario where senders can’t afford to block.
Pulse has fixed payload size (32 bit value, 8 bit code)
Pulse initialization/send/receive
  • SIGEV_PULSE_INIT(&event, coid, priority, pulse_code, pulse_value);
    • If you want the thread that receives the pulse to run at the initial priority of the process, set priority to SIGEV_PULSE_PRIO_INHERIT.
  • Int MsgSendPulse(coid, priority, pulse_code, pulse_value)
  • Int MsgRecivePulse(chid, &pulse, sizeof(pulse), struct _msg_info *info);  // info should be NULL for pulse.
  • Int MsgReceive(chi, &msg, bytes, struct _msg_info *info);       // a return of “0” means a pulse is received; if >0, a normal message.

Priority Inversion: a low-piority thread consumes all available CPU time, even though a higher-priroity thread is ready to run.

To avoid it, the solution is Priority inheritance, or Message-driven priority: A server process receives messages & pulses in priority order. the receiving thread within the server process inherits the priority of the sender thread; when it receives messages (for pulses, receiving thread will run at the priority of the pulse).

Assume there are 3 threads, t1 (priority 13), t2 (priority 10), server thread (priority 22),

  • If the sender(t2) is of lower priority (10) than the receiver (22), the effective priority of the server thread changes when the message is received (on MsgReceive).
    • the sender(t2)’s work is done at priority 10.
    • If another higher priority thread(t1) is running when t2 calls MsgSend(), it will not preempted by t2.
  • If the sender(t1) is of higher priority(13) than the receiver(10), the effective priority of the server thread changes on MsgSend().
    •  If multiple clients are SEND blocked, server will inherit the priority the higher client.
  • After the server reply to t1 and t2, its priority will still be 13, and receive blocked.
  • Priority inheritance can be turned off by channel flag _NTO_CHF_FIXED_PRIORITY. If the channel is created with _NTO_CHF_FIXED_PRIORITY, receiving threads won’t change priorities to those of the sending threads (does it work for pulses???)
  • For pulses (an example of pulse priority)
    • Pulse have a priority value while being initialized. If you want the thread that receives the pulse to run at the initial priority of the process, set sigev_priority to SIGEV_PULSE_PRIO_INHERIT (note: not the priority of the thread which sends the pulse, but the initial priority of the process!???)
    • Sender can specify the priority of the pulse, or –1 to use the priority of the calling thread.
#include <sys/neutrino.h>
#include <sys/netmgr.h>
int chid = ChannelCreate(unsigned flags);
Int coid = ConnectAttach(node_id, pid, chid, 0 | _NTO_SIDE_CHANNEL, flags);
Struct sigevent event;
SIGEV_PULSE_INIT(&event, coid, prio, MY_PULSE, NULL);
In a thread,
struct _pulse pulse;
If(MsgReceivePulse(chid, &pulse, sizeof(pulse), NULL) != -1) {  //similar to MsgReceive(), but only receives pulses.
    Switch(pulse.code) {
        Case MY_PULSE:
In the other thread,
MsgSendPulse(coid, -1, MY_PULSE, 0);