{"id":1694,"date":"2015-01-16T00:00:00","date_gmt":"2015-01-16T08:00:00","guid":{"rendered":"http:\/\/192.168.3.4\/?p=1694"},"modified":"2018-01-09T06:49:49","modified_gmt":"2018-01-09T14:49:49","slug":"rpi-image-processing","status":"publish","type":"post","link":"https:\/\/www.cloudacm.com\/?p=1694","title":{"rendered":"RPi image processing"},"content":{"rendered":"

Introduction – processing images with ImageMagick<\/strong><\/p>\n

I first\u00a0posted about ImageMagick in early June, last year. \u00a0Then late\u00a0November, I covered the steps to install it and use it on RPi. \u00a0The features of ImageMagick are fairly extensive, I won’t go into all of them here. \u00a0As a guide, I recommend\u00a0Michael Still’s book titled “The Definitive Guide to ImageMagick<\/a>” through APress publishing. \u00a0For additional information, you may want to look at\u00a0Sohail Salehi’s book titled “ImageMagick Tricks<\/a>” from Packt Publishing.<\/p>\n

ImageMagick<\/a>\u00a0is defined on its\u00a0website.<\/p>\n

“…is designed for batch processing of images. That is, it allow(s) you to combine image processing operations in a script (shell, DOS, Perl, PHP, etc.) so the operations can be applied to many images, or as a sub-system of some other tool, such as a Web application, video processing tool, panorama generator, etc. It is not a GUI image editor…”<\/em><\/p>\n

It seems that image processing is a perfect fit for ImageMagick on the RPi. \u00a0I’ve been web scraping images since last quarter. \u00a0These raw images are a perfect resource to process for data analysis. \u00a0Real time processing lends itself to other uses as well, one being augmented reality.<\/p>\n

In this post, I’ll cover the basics of ImageMagick. \u00a0We’ll see examples of image processing to create video streams. \u00a0I’ll also go over how to extract data from images to represent a condition, which we can plot over time. \u00a0We’ll also see how to reprocess image data into new images and video for an enhanced visual experience. \u00a0We have a lot cover, so lets start.<\/p>\n

Purpose – Seeing the light<\/strong><\/p>\n

I’ve covered image processing before, but this will be the first time that we will actually manipulate and quantify data sets from images. \u00a0We will do this so the\u00a0images become an extensible\u00a0daylight, weather, or\u00a0traffic sensor. \u00a0The images contain visual data that is easy for humans to understand, but the details often get overlooked. \u00a0This where the RPi comes in. \u00a0It can processes the images into data sets. \u00a0This is useful because it gives us a better understanding of the image content. \u00a0Not only that, I gives us the ability to program the RPi to perform an\u00a0action based on a condition. \u00a0In essence, we will program our RPi to do something in response to what it sees.<\/p>\n

Details – It is in there, you just need to look harder<\/strong><\/p>\n

First things first, I would like to revisit the process of converting images to a format that will allow video creation. \u00a0Since images gathered from cameras typically are in the JPG format, we won’t\u00a0have much trouble in converting the images to video. \u00a0However, images that contain\u00a0few color variations usually are in a GIF or PNG format. \u00a0I’ve run into issues with background transparencies. \u00a0So, here is the command to use when converting both GIF and PNG formats to JPG.<\/p>\n

\r\n\r\nconvert -background white -flatten  -compress none -quality 100100  SourceImage DestinationImage\r\n<\/code>\r\n<\/pre>\n
This can be a problem if you’re converting several files in a directory. \u00a0For that, I use this batch command\u00a0when converting images on my Windows machine.<\/p>\n
\r\n\r\n@echo off\r\n\r\ndir *.gif \/b > FileList.txt\r\n\r\nFOR \/F \"tokens=*\" %%A IN (FileList.txt) DO (\r\nconvert -background white -flatten -compress none -quality 100100 %%A %%A.jpg\r\n)\r\n\r\nexit\r\n<\/code>\r\n<\/pre>\n
In this example we specify the background transparency to the color “white”, then we flatten the image. \u00a0The thing about JPG that is worth mentioning is that it is a lossy compression algorithm. \u00a0Since I don’t want to loose any image data, I specify that no compression be used and that the quality be %100 for all options.<\/p>\n
Now that the images are available in the correct format, we can continue on to process them into video streams. \u00a0This command should do the trick.<\/p>\n
\r\n\r\nmencoder mf:\/\/*.jpg -mf fps=60 -o DestinationVideo.avi -ovc lavc -lavcopts vcodec=mpeg4\r\n<\/code>\r\n<\/pre>\n
In this command we’re taking all of the JPG files in the current directory and converting them to a MPEG4 video with the AVI extension. \u00a0The output video will be at 60 frames per second.<\/p>\n
With the revisited image and video creation process out of the way, I want to show a simple procedure to show a condition unique to the web scraped images. \u00a0In this next example I will plot a graph from the images. \u00a0This graph will represent day and night time conditions.<\/p>\n
\"FileSizePlot\"<\/a><\/p>\n
 <\/p>\n
 <\/p>\n
 <\/p>\n
 <\/p>\n
This graph was created using the file sizes of each image over a period of time. \u00a0There are\u00a0discrepancies with this method, but it does reveal something interesting. \u00a0The image compression is greater when most of the image scene is uniform. \u00a0In contrast, when the image scene has a diversity of pixel values, the compression is lower. \u00a0In the graph above, we can also see the peak variations. \u00a0This is an attribute of weather. \u00a0During overcast days, the pixel diversity is lower than it is on clear sunny days.<\/p>\n
At this point you may be asking yourself, what does this have to do with ImageMagick? \u00a0The problem with the data analysis we’ve done so far is it lacks detail. \u00a0The data isn’t a direct observation of a condition\u00a0and this\u00a0is a valid concern. \u00a0We only have a graph of the file sizes over time, which is an observation of the JPG compression. \u00a0There are several conditions that will influence the file size. \u00a0GIF and PNG compression will not show the same variations of file size, so our earlier method will not work. \u00a0 With ImageMagick, we can make observations at the pixel level. \u00a0Here is were the magic is.<\/p>\n
In this image, we can visually see the road conditions plotted. \u00a0It’s simple and to the point. \u00a0However, we can not tell how these conditions change over time.<\/p>\n
\"WSDOT_Bridges_20150112200002\"<\/a><\/p>\n
 <\/p>\n
 <\/p>\n
 <\/p>\n
 <\/p>\n
 <\/p>\n
 <\/p>\n
This is a\u00a0problem that ImageMagick will solve for us. \u00a0We will be able to use ImageMagick to\u00a0plot a\u00a0condition of a specific point over time.<\/p>\n
The first step in the process is to get the image then select the pixels to analize. \u00a0In this example, we want to quantify the volume of west bound traffic on the 520 bridge. The image scrapped from the WSDOT website has a dimension of 780 x 572 and is in\u00a0a GIF format. The pixels that contain data pertinent\u00a0to our needs\u00a0are\u00a0located at 238\u00a0x\u00a0235 and they span 4 x 4. With that information, we can crop the image into a new file that only contains the pixels we what data from. One thing to keep in mind is JPG crops are simple. GIF images on the otherhand can pose issues if the transparency layer is not removed. For that reason, I have the same command listed below to show the differences.<\/p>\n
\r\n\r\nconvert -crop 4x4+238+235 InputImage.jpg CroppedOutputImage.jpg\r\nconvert -crop 4x4+238+235 +repage InputImage.gif CroppedOutputImage.gif\r\n<\/code>\r\n<\/pre>\n
Now that we have the pixels isolated, we can run a histogram analysis on the cropped image. Here is the command to do that.<\/p>\n
\r\n\r\nconvert CroppedOutputImage.gif -format %c histogram:info:TextFile.txt\r\n<\/code>\r\n<\/pre>\n
This generates a text file that will tally the number of pixels that are of a certain color. We are looking for 6 specific colors which are green, yellow, red, black, grey, and white. The histogram report will come back with the following text.<\/p>\n
\r\n\r\n16: (32,224,64) #20E040 srgb(32,224,64)\r\n<\/code>\r\n<\/pre>\n
From here we can see that all 16 of the pixels are the same color, #20E040, which is green. Now that we have a text based reading of the pixels, we can associate it with a value. This will involve some text parsing<\/a> and processing, something of which python\u00a0is well suited for.<\/p>\n
\r\n\r\nmyfile = open(\"\/home\/pi\/cacti_scripts\/TextFile.txt\").read().replace('\\n','')\r\ncutfile = myfile.partition('#')[2]\r\nhead, sep, tail = cutfile.partition(' ')\r\ntext_file = open(\"\/home\/pi\/cacti_scripts\/HEXReading.txt\", \"w\")\r\ntext_file.write(head)\r\ntext_file.close()\r\n<\/code>\r\n<\/pre>\n
The great thing about processing the text information is now we can port it over to Cacti to graph over time. To do that, we’ll run a text format script in python and setup our output so Cacti can read it.<\/p>\n
\r\n\r\nif head in ('000000'):\r\n      cactivalue = open(\"\/home\/pi\/cacti_scripts\/HEXReading.txt\").read().replace('000000','traffic:4')\r\nelif head in ('FF0000'):\r\n      cactivalue = open(\"\/home\/pi\/cacti_scripts\/HEXReading.txt\").read().replace('FF0000','traffic:3')\r\nelif head in ('FFFF00'):\r\n      cactivalue = open(\"\/home\/pi\/cacti_scripts\/HEXReading.txt\").read().replace('FFFF00','traffic:2')\r\nelif head in ('20E040'):\r\n      cactivalue = open(\"\/home\/pi\/cacti_scripts\/HEXReading.txt\").read().replace('20E040','traffic:1')\r\nelse:\r\n      cactivalue = 'traffic:0'\r\n\r\nprint cactivalue\r\n<\/code>\r\n<\/pre>\n
Now is a perfect time to break away from image processing and revisit Cacti. You can view my earlier posts on Cacti<\/a>, especially on the light sensor readings using RPi. Here we’ll need to create a new device and data source to graph the data. \u00a0In that post I reference another blog that did the leg work, you can find the document here<\/a>.<\/p>\n
First we’ll create our Data Input Method by opening the Cacti console on our RPi and selecting DIM under the Collection Methods section. \u00a0Click the Add link on the upper right corner to start a new DIM. \u00a0Enter the name of the DIM, we’ll use “West Bound 520”. \u00a0Set input type to “Script\/Command”. \u00a0Then we’ll enter the input string which is the path and python script that Cacti will use to get the current reading. \u00a0Yours may differ, my input string is “\/var\/www\/cacti\/scripts\/wb520.py” \u00a0Clicking Create will add some more options we’ll need to enter before continuing on.<\/p>\n
In the Output Fields we’ll enter “traffic” for the Field Output, since this will be the reading identifier which originates from our “wb520.py” script. \u00a0Next we can enter a friendly name, which we’ll use “WB520Traffic”. \u00a0Now we are done with that in Cacti, lets go on to our Data Sources.<\/p>\n
Click Data Sources from the Management section and click Add in the top right. \u00a0Leave the Data Template blank and choose the RPi from the pull down. \u00a0It should have either 127.0.0.1 or the IP you assigned to it in parenthesis. \u00a0Clicking the Create button will bring us to more options.<\/p>\n
We’ll enter in “WB520Data” for the Data Source name. \u00a0The Data Source Path should be left blank. The Data Input Method is the script we created, “West Bound 520”. \u00a0We’ll leave the step at 300 seconds, since the images won’t be updated any faster.<\/p>\n
The internal Data Source Name has to match the entry we set in the Output Field and “wb520.py” script. \u00a0In our case, this is “traffic”. \u00a0Lastly, we enter our max and min values we expect to read followed by the heartbeat setting. \u00a0The heartbeat is a value in seconds that will let Cacti know when to enter a reading of unknown if that much time has passed since a valid reading.<\/p>\n
Now our Output Field will be visible, be sure it show “traffic – WB520”. \u00a0Click Save and now we’re ready to setup our graph template.<\/p>\n
Click Graph Management, then the Add link at the top right. \u00a0Choose None from the Selected Graph Template and for the Host choose the RPi from the pull down. \u00a0We’ll set the title to “West Bound 520 Traffic”. \u00a0The next thing\u00a0to set\u00a0is the Vertical Label, which we’ll put “intensity”. \u00a0We’re done, so we can click Create.<\/p>\n
Now we should have a Graph Items section added to the window.Lets start by clicking Add in that new section. \u00a0Now we can select\u00a0our data source, in our case it is “WB520Data (traffic)”. \u00a0I’ll be using “C00000” for the graph color. \u00a0Next we’ll choose “AREA” as the graph type. \u00a0The only other thing I’ll set is “Traffic” in the Text Format section, which is used in the legend.<\/p>\n
That’s it! \u00a0 Now we have data being entered into Cacti…woohoo!<\/p>\n
\"WB520Graph\"<\/a>Here is the complete source script that Cacti uses to download, process, and read.<\/p>\n
\r\n\r\n#!\/usr\/bin\/python\r\n\r\nimport urllib\r\nimport os\r\n\r\nurllib.urlretrieve(\"http:\/\/images.wsdot.wa.gov\/nwflow\/flowmaps\/bridges.gif\", filename=\"\/home\/pi\/cacti_scripts\/WSDOT_Bridges.gif\")\r\n\r\nos.system(\"convert -crop 4x4+238+235 +repage \/home\/pi\/cacti_scripts\/WSDOT_Bridges.gif \/home\/pi\/cacti_scripts\/cropped.gif\")\r\nos.system(\"convert \/home\/pi\/cacti_scripts\/cropped.gif -format %c histogram:info:\/home\/pi\/cacti_scripts\/TextFile.txt\")\r\n\r\nmyfile = open(\"\/home\/pi\/cacti_scripts\/TextFile.txt\").read().replace('\\n','')\r\ncutfile = myfile.partition('#')[2]\r\nhead, sep, tail = cutfile.partition(' ')\r\ntext_file = open(\"\/home\/pi\/cacti_scripts\/HEXReading.txt\", \"w\")\r\ntext_file.write(head)\r\ntext_file.close()\r\n\r\nif head in ('000000'):\r\n      cactivalue = open(\"\/home\/pi\/cacti_scripts\/HEXReading.txt\").read().replace('000000','traffic:4')\r\nelif head in ('FF0000'):\r\n      cactivalue = open(\"\/home\/pi\/cacti_scripts\/HEXReading.txt\").read().replace('FF0000','traffic:3')\r\nelif head in ('FFFF00'):\r\n      cactivalue = open(\"\/home\/pi\/cacti_scripts\/HEXReading.txt\").read().replace('FFFF00','traffic:2')\r\nelif head in ('20E040'):\r\n      cactivalue = open(\"\/home\/pi\/cacti_scripts\/HEXReading.txt\").read().replace('20E040','traffic:1')\r\nelse:\r\n      cactivalue = 'traffic:0'\r\n\r\nprint cactivalue\r\n<\/code>\r\n<\/pre>\n
Since the grueling work is now behind us, we can move forward to the last item, which will be video from enhanced images. \u00a0We’ll do a simple normalize command in ImageMagick to create a new series of images. \u00a0Next we’ll process these to create a video stream. \u00a0Lets wrap this topic up. \u00a0Here is the command to run.<\/p>\n
\r\n\r\nconvert -normalize InputImage.jpg OutputImage.jpg\r\n<\/code>\r\n<\/pre>\n
In my archive, I have around 10,000 images to process through. \u00a0It\u00a0took\u00a0some time to complete, 8 hours to be more precise. \u00a0Once they were done normalizing, I went ahead and processed them into a video stream. \u00a0Here are the results. \u00a0The left is the normalized images and the right are the original images.<\/p>\n