Image Processing http://simonsilk.com/category/category/image-processing en Pixel Format Conversions http://simonsilk.com/content/simonsilk/2011-mar/pixel-format-conversions <span class="field field--name-title field--type-string field--label-hidden">Pixel Format Conversions</span> <div class="clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item"><p>I recently found myself having to do a few interesting conversions in C++ to pass images between different formats and/ or libraries, including OpenCV, MATLAB, and some unusual custom formats. I'm going to post some code snippets here that do some of these conversions as I suspect they may be useful for others.</p> <p> </p> <p>Keep in mind that, generally, image data is always stored in one long array, and you have a pointer to the beginning of that array. So, the question is, as you increment your way through the array contents, in what order are you traversing the image's data? E.g. are the first three elements the RGB values of the first pixel in the top left corner of the image? Or are they the R components of the first three pixels moving right along the top row of the image? Or something else entirely?</p> <p> </p> <!--break--><p>As it turns out, this depends on what library/ API/ SDK/ whatever you are using to handle images. You may often want to use multiple image types in the same project so that you can leverage the capabilities of different libraries or programs, for example using ImageMagick to load some obscure image format, then transferring the loaded image data into an OpenCV Mat structure so you can use OpenCV's routines on it. As a results, being able to quickly convert between these different pixel formats is very handy.</p> <p> </p> <p>A few handy defiintions will make the conversions I'm describing a litte more clear.</p> <p> </p> <p><strong><a href="http://en.wikipedia.org/wiki/Row-major_order" target="_blank">Row-major order</a></strong> means that as we increment through the array, we first move along the top-most row of the matrix (image), then along columns, i.e. when we get to the last entry on the first row and increment one more place, we are at the first pixel of the next horizontal row. This is the most common way to handle matrices in C and C++. <strong>Column-major</strong> should now be self-explanatory; as we increment through an array, we are first moving down the left-most column of the matrix, then along rows.</p> <p> </p> <p><strong>Interleaved </strong>color channels (or color planes) means that the first thing we increment through is the color channels of the current pixel, then through pixels, i.e. for interleaved RGB data, the data is ordered R1, G1, B1, R2, G2, B2..., where the numbers index pixels. <strong>De-interleaved</strong> data seperates the color planes so that we first increment through all the pixels of the first color plane before moving to the next color plane, i.e. we would go through R1, R2, R3..., G1, G2, G3,...., B1, B2, B3.</p> <p> I'll continue posting new conversions here as I find need for them in my own work, but in the meantime, if you don't see the exact conversion here that you want, the ones I've provided should at least help a bit in figuring out how to do your own, and you can always chain multiple conversions together.</p> <p> </p> <h2>MATLAB to RGB</h2> <p><strong>...or column-major de-interleaved RGB to row-major interleaved RGB. </strong></p> <p>This may not be the most apt name for this conversion, but here's the idea. I was writing a MEX file so that I could call some C++ image processing code from within MATLAB. That code was working with interleaved RGB data in row-major order. However, mex files receive inputs from MATLAB as <a href="http://www.mathworks.com/help/techdoc/apiref/mxarray.html" target="_blank">mxArrays</a>, which <a href="http://www.mathworks.com/support/tech-notes/1600/1605.html#mxArray" target="_blank">use column-major</a> de-interleaved data. The column major ordering of MATLAB's data is a throwback to when it was originally written in Fortran.</p> <p> </p> <p>So, before using the data passed from MATLAB, it needs to be rearranged. This can be done as follows, where I'm assuming there could be multiple input images, indexed by <em>int img</em>. Also note that since we don't necessarily know the data type of the input ahead of time, I've included a check for this, which changes the cast of the input data pointer so it gets dereferenced correctly. Here I'm just allowing for <em>double</em> or <em>uint8</em>.</p> <p> </p> <p> </p> <p> </p> <p></p><div class="geshifilter"><pre class="cpp geshifilter-cpp" style="font-family:monospace;"><span style="color: #666666;">// Make array to hold rearranged data </span> pInput_RGB <span style="color: #000080;">=</span> <span style="color: #0000dd;">new</span> <span style="color: #0000ff;">char</span><span style="color: #008000;">[</span>height<span style="color: #000040;">*</span>width<span style="color: #000040;">*</span>depth<span style="color: #008000;">]</span><span style="color: #008080;">;</span> <span style="color: #666666;">// Get pointer to input data </span> <span style="color: #0000ff;">double</span> <span style="color: #000040;">*</span>mxPtr <span style="color: #000080;">=</span> mxGetPr<span style="color: #008000;">(</span>prhs<span style="color: #008000;">[</span>img<span style="color: #008000;">]</span><span style="color: #008000;">)</span><span style="color: #008080;">;</span> <span style="color: #666666;">// img indexes input images </span> <span style="color: #666666;">//Rearrange data from MATLAB's column-major, de-interleaved RGB order to C-standard row-major interleaved RGB order: </span> <span style="color: #0000ff;">int</span> idx_RGB_RC<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> <span style="color: #666666;">// indexes target data format </span> <span style="color: #0000ff;">int</span> idx_CR_RGB<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> <span style="color: #666666;">// indexes MATLAB's format (column, row, color channel) </span> <span style="color: #666666;">// Don't know ahead of time the data type of input. Will accept double or uint8 </span> <span style="color: #0000ff;">if</span><span style="color: #008000;">(</span>mxGetClassID<span style="color: #008000;">(</span>prhs<span style="color: #008000;">[</span>img<span style="color: #008000;">]</span><span style="color: #008000;">)</span><span style="color: #000080;">==</span>mxDOUBLE_CLASS<span style="color: #008000;">)</span><span style="color: #008000;">{</span> <span style="color: #666666;">// Cast incoming data ptr to double </span> <span style="color: #0000ff;">for</span><span style="color: #008000;">(</span>j<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> j<span style="color: #000080;">&lt;</span>height<span style="color: #008080;">;</span> j<span style="color: #000040;">++</span><span style="color: #008000;">)</span><span style="color: #008000;">{</span> <span style="color: #0000ff;">for</span><span style="color: #008000;">(</span>i<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> i<span style="color: #000080;">&lt;</span>width<span style="color: #008080;">;</span> i<span style="color: #000040;">++</span><span style="color: #008000;">)</span><span style="color: #008000;">{</span> <span style="color: #0000ff;">for</span><span style="color: #008000;">(</span>k<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> k<span style="color: #000080;">&lt;</span>depth<span style="color: #008080;">;</span> k<span style="color: #000040;">++</span><span style="color: #008000;">)</span><span style="color: #008000;">{</span> idx_RGB_RC <span style="color: #000080;">=</span> <span style="color: #008000;">(</span>j<span style="color: #000040;">*</span>width<span style="color: #000040;">*</span>depth<span style="color: #008000;">)</span><span style="color: #000040;">+</span><span style="color: #008000;">(</span>i<span style="color: #000040;">*</span>depth<span style="color: #008000;">)</span><span style="color: #000040;">+</span>k<span style="color: #008080;">;</span> idx_CR_RGB <span style="color: #000080;">=</span> <span style="color: #008000;">(</span>k<span style="color: #000040;">*</span>height<span style="color: #000040;">*</span>width<span style="color: #008000;">)</span><span style="color: #000040;">+</span><span style="color: #008000;">(</span>i<span style="color: #000040;">*</span>height<span style="color: #008000;">)</span><span style="color: #000040;">+</span>j<span style="color: #008080;">;</span> <span style="color: #008000;">(</span><span style="color: #008000;">(</span><span style="color: #0000ff;">char</span><span style="color: #000040;">*</span><span style="color: #008000;">)</span>pInput_RGB<span style="color: #008000;">)</span><span style="color: #008000;">[</span>idx_RGB_RC<span style="color: #008000;">]</span> <span style="color: #000080;">=</span> <span style="color: #008000;">(</span><span style="color: #0000ff;">char</span><span style="color: #008000;">)</span><span style="color: #008000;">(</span><span style="color: #008000;">(</span> <span style="color: #008000;">(</span> <span style="color: #008000;">(</span><span style="color: #0000ff;">double</span><span style="color: #000040;">*</span><span style="color: #008000;">)</span><span style="color: #008000;">(</span>mxGetPr<span style="color: #008000;">(</span>prhs<span style="color: #008000;">[</span>img<span style="color: #008000;">]</span><span style="color: #008000;">)</span><span style="color: #008000;">)</span> <span style="color: #008000;">)</span> <span style="color: #008000;">[</span>idx_CR_RGB<span style="color: #008000;">]</span><span style="color: #008000;">)</span><span style="color: #008000;">)</span><span style="color: #008080;">;</span> <span style="color: #008000;">}</span> <span style="color: #008000;">}</span> <span style="color: #008000;">}</span> <span style="color: #008000;">}</span> <span style="color: #0000ff;">else</span> <span style="color: #0000ff;">if</span> <span style="color: #008000;">(</span>mxGetClassID<span style="color: #008000;">(</span>prhs<span style="color: #008000;">[</span>img<span style="color: #008000;">]</span><span style="color: #008000;">)</span><span style="color: #000080;">==</span>mxUINT8_CLASS<span style="color: #008000;">)</span><span style="color: #008000;">{</span> <span style="color: #666666;">// Cast incoming data ptr to char </span> <span style="color: #0000ff;">for</span><span style="color: #008000;">(</span>j<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> j<span style="color: #000080;">&lt;</span>height<span style="color: #008080;">;</span> j<span style="color: #000040;">++</span><span style="color: #008000;">)</span><span style="color: #008000;">{</span> <span style="color: #0000ff;">for</span><span style="color: #008000;">(</span>i<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> i<span style="color: #000080;">&lt;</span>width<span style="color: #008080;">;</span> i<span style="color: #000040;">++</span><span style="color: #008000;">)</span><span style="color: #008000;">{</span> <span style="color: #0000ff;">for</span><span style="color: #008000;">(</span>k<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> k<span style="color: #000080;">&lt;</span>depth<span style="color: #008080;">;</span> k<span style="color: #000040;">++</span><span style="color: #008000;">)</span><span style="color: #008000;">{</span> idx_RGB_RC <span style="color: #000080;">=</span> <span style="color: #008000;">(</span>j<span style="color: #000040;">*</span>width<span style="color: #000040;">*</span>depth<span style="color: #008000;">)</span><span style="color: #000040;">+</span><span style="color: #008000;">(</span>i<span style="color: #000040;">*</span>depth<span style="color: #008000;">)</span><span style="color: #000040;">+</span>k<span style="color: #008080;">;</span> idx_CR_RGB <span style="color: #000080;">=</span> <span style="color: #008000;">(</span>k<span style="color: #000040;">*</span>height<span style="color: #000040;">*</span>width<span style="color: #008000;">)</span><span style="color: #000040;">+</span><span style="color: #008000;">(</span>i<span style="color: #000040;">*</span>height<span style="color: #008000;">)</span><span style="color: #000040;">+</span>j<span style="color: #008080;">;</span> <span style="color: #008000;">(</span><span style="color: #008000;">(</span><span style="color: #0000ff;">char</span><span style="color: #000040;">*</span><span style="color: #008000;">)</span>pInput_RGB<span style="color: #008000;">)</span><span style="color: #008000;">[</span>idx_RGB_RC<span style="color: #008000;">]</span> <span style="color: #000080;">=</span> <span style="color: #008000;">(</span><span style="color: #0000ff;">char</span><span style="color: #008000;">)</span><span style="color: #008000;">(</span><span style="color: #008000;">(</span> <span style="color: #008000;">(</span> <span style="color: #008000;">(</span><span style="color: #0000ff;">char</span><span style="color: #000040;">*</span><span style="color: #008000;">)</span><span style="color: #008000;">(</span>mxGetPr<span style="color: #008000;">(</span>prhs<span style="color: #008000;">[</span>img<span style="color: #008000;">]</span><span style="color: #008000;">)</span><span style="color: #008000;">)</span> <span style="color: #008000;">)</span> <span style="color: #008000;">[</span>idx_CR_RGB<span style="color: #008000;">]</span><span style="color: #008000;">)</span><span style="color: #008000;">)</span><span style="color: #008080;">;</span> <span style="color: #008000;">}</span> <span style="color: #008000;">}</span> <span style="color: #008000;">}</span> <span style="color: #008000;">}</span></pre></div> <p> </p> <h2>RGB to ARGB</h2> <p>Some publicly released research code I was using recently required images stored in an unusual integer ARGB format where each pixel was represented by a single integer (4 bytes); bytes two through four were the RGB channels, and the first byte was an (unused as it turns out) <a href="http://en.wikipedia.org/wiki/Alpha_compositing" target="_blank">alpha channel</a>. This is of course basically the more common <a href="http://en.wikipedia.org/wiki/RGBA_color_space" target="_blank">RGBA</a> format, but with the alpha channel moved to first place for some reason.</p> <p> </p> <p>Anyway, the conversion is pretty straightforward; you need to create space for the converted data, then use char pointers to access each of the four bytes making up a pixel individually. In the following snippet I've assumed the input is accessed by a <em>char </em>pointer <em>pInput_RGB</em>, the output is an <em>unsigned int </em>pointer <em>pInput_ARGB</em>. I've also just hard-coded the fact that there are four channels in the ARGB format.</p> <p> </p> <p></p><div class="geshifilter"><pre class="cpp geshifilter-cpp" style="font-family:monospace;"><span style="color: #0000ff;">unsigned</span> <span style="color: #0000ff;">int</span><span style="color: #000040;">*</span> pInput_ARGB <span style="color: #000080;">=</span> <span style="color: #0000dd;">new</span> <span style="color: #0000ff;">unsigned</span> <span style="color: #0000ff;">int</span><span style="color: #008000;">[</span>height<span style="color: #000040;">*</span>width<span style="color: #008000;">]</span><span style="color: #008080;">;</span>   <span style="color: #666666;">// Convert from RGB to ARGB:</span> <span style="color: #0000ff;">int</span> idx_RGB<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span>, idx_ARGB<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> <span style="color: #0000ff;">for</span><span style="color: #008000;">(</span>j<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> j<span style="color: #000080;">&lt;</span>height<span style="color: #008080;">;</span> j<span style="color: #000040;">++</span><span style="color: #008000;">)</span><span style="color: #008000;">{</span> <span style="color: #0000ff;">for</span><span style="color: #008000;">(</span>i<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> i<span style="color: #000080;">&lt;</span>width<span style="color: #008080;">;</span> i<span style="color: #000040;">++</span><span style="color: #008000;">)</span><span style="color: #008000;">{</span> <span style="color: #0000ff;">for</span><span style="color: #008000;">(</span>k<span style="color: #000080;">=</span><span style="color: #0000dd;">0</span><span style="color: #008080;">;</span> k<span style="color: #000080;">&lt;</span>depth<span style="color: #008080;">;</span> k<span style="color: #000040;">++</span><span style="color: #008000;">)</span><span style="color: #008000;">{</span> idx_RGB <span style="color: #000080;">=</span> <span style="color: #008000;">(</span>j<span style="color: #000040;">*</span>width<span style="color: #000040;">*</span>depth<span style="color: #008000;">)</span><span style="color: #000040;">+</span><span style="color: #008000;">(</span>i<span style="color: #000040;">*</span>depth<span style="color: #008000;">)</span><span style="color: #000040;">+</span>k<span style="color: #008080;">;</span> idx_ARGB <span style="color: #000080;">=</span> <span style="color: #008000;">(</span>j<span style="color: #000040;">*</span>width<span style="color: #000040;">*</span><span style="color: #0000dd;">4</span><span style="color: #008000;">)</span><span style="color: #000040;">+</span><span style="color: #008000;">(</span>i<span style="color: #000040;">*</span><span style="color: #0000dd;">4</span><span style="color: #008000;">)</span><span style="color: #000040;">+</span>k<span style="color: #000040;">+</span><span style="color: #0000dd;">1</span><span style="color: #008080;">;</span> <span style="color: #008000;">(</span><span style="color: #008000;">(</span><span style="color: #0000ff;">char</span><span style="color: #000040;">*</span><span style="color: #008000;">)</span>pInput_ARGB<span style="color: #008000;">)</span><span style="color: #008000;">[</span>idx_ARGB<span style="color: #008000;">]</span> <span style="color: #000080;">=</span> <span style="color: #008000;">(</span><span style="color: #008000;">(</span><span style="color: #0000ff;">char</span><span style="color: #000040;">*</span><span style="color: #008000;">)</span>pInput_RGB<span style="color: #008000;">)</span><span style="color: #008000;">[</span>idx_RGB<span style="color: #008000;">]</span><span style="color: #008080;">;</span> <span style="color: #008000;">}</span> <span style="color: #008000;">}</span> <span style="color: #008000;">}</span></pre></div> <p> </p> <p> </p> </div> <span class="field field--name-uid field--type-entity-reference field--label-hidden"><span lang="" about="/users/simonsilk" typeof="schema:Person" property="schema:name" datatype="">SimonSilk</span></span> <span class="field field--name-created field--type-created field--label-hidden">Sun, 03/20/2011 - 23:51</span> <div class="field field--name-tags field--type-entity-reference field--label-hidden field--entity-reference-target-type-taxonomy-term clearfix field__items"> <div class="field__item"><a href="/category/tags/snippet" hreflang="en">Snippet</a></div> </div> <div class="field field--name-category field--type-entity-reference field--label-hidden field--entity-reference-target-type-taxonomy-term clearfix field__items"> <div class="field__item"><a href="/category/category/image-processing" hreflang="en">Image Processing</a></div> <div class="field__item"><a href="/category/category/programming" hreflang="en">Programming</a></div> </div> <section class="field field--name-comment field--type-comment field--label-above comment-wrapper"> <h2 class="title comment-form__title">Add new comment</h2> <drupal-render-placeholder callback="comment.lazy_builders:renderForm" arguments="0=node&amp;1=22&amp;2=comment&amp;3=comment" token="8G-0ziIYgIKVNZP_lsiweU7U_jtelpkeubzWPKGyBMY"></drupal-render-placeholder> </section> Mon, 21 Mar 2011 02:51:54 +0000 SimonSilk 22 at http://simonsilk.com