Far Watch WATCHER

This page describes FW003, "Far Watch Watcher".

"Far Watch Watcher" is a utility which increases the benefit you get if you have set up an Arduserver... which can cost as little as $16... or have a FarWatch webserver. Or some other web server which includes two lines that are there for the wants of FarWatchWatcher.

Arduservers and FarWatch servers are webservers which have sensors attached to them. They are real, physical "things", set up in a specific place... and connected to sensors. Once they are set up, anyone on the planet (or in the ISS!) can see, across the internet, what readings are coming in on those sensors. The interested person needs nothing more than a simple web browser.

In September 2018, I did more work with sensor reporting webservers built in ESP8266's ($16, plus cost of sensor), and did created a new generation of FarWatchWatchers. Unfortunately, none of that is written up well... but if you don't mind struggling a bit, there is free software for downloading.

This page starts with "an ad", and then goes into the technical detail of how you would actually do it. Don't let the long "doing it" put you off. Yes, the program does have to be "set up"... but it really isn't as bad as all the text might make you fear. And once set up, it Just Works!

You can try FarWatchWatcher for free, but please do not pass it on. Tell interested parties about this page, instead, please.

You can see FarWatchWatcher working by using it to watch one of my FarWatch servers. (My Arduservers are FarWatch servers, too.)

They are hobbyist-level installations, so don't waste a lot of time if your web browser can't get through at the moment, but for what it is worth, either of these should respond. Both are in Arduservers, the second one in a $16 ESP8266.

http://mon277rr.dyndns.org:1034/ (A rather dull page! Monitors conditions at a location between New York and Boston, USA.) Two temperatures, two humidities.

http://mon277rr.dyndns.org:1200/ (A rather fancier page! Monitors: P0:Indoor tture, P1: Wind activity (speed), P2: Outdoor tture, P3: nothing! Again, at a location between New York and Boston, USA.)

If you think that's fun... or you may go about this the other way around... you will eventually want to set up an Arduserver ($16!) (or FarWatch and DS025) of your own, so you can watch your own data! (See the FarWatch overview page.)

The servers give you a webpage that tells you what a device somewhere is seeing on sensors attached to it. Tells you across the internet, where ever you and a simple browser happen to be. (And, although it is nothing to do with FarWatchWatcher, still with just a browser, you can turn things attached to an Arduserver on and off.)

FarWatchWATCHER FW003 automatically harvests the data the server has on offer. It draws a graph of the readings, and records them in a spreadsheet- friendly data file. (Some of my other FarWatchWatchers will raise an alarm if, say, a temperature monitored by the server falls below a figure chosen by you. FW003 doesn't... yet.. offer that.)

The first on the right shows the raw incoming data. The second is a snippet from the data file that is created.

The details of how you interpret graph, raw data, and data log are all covered in greater detail further down the page.

How you would actually get a FarWatch Watcher up and running...

Still reading? Great! You have an Arduserver running? (or one of mine is responding.) Time to download and try FW003, FarWatchWatcher...

Click here to download zip file with all you need. THE PROGRAM IS UNDER DEVELOPMENT just now. It WILL read graph and record temperatures and pulse counts, though. Don't hesitate to get in touch, should you have questions.

Establish (or find an established) webserver, e.g. an Arduserver or FarWatch installation, which has one of the following at the start of the line in it somewhere...

hhhhsttthhhhsttt

or

hhhhsttthhhhstttb

("stuff" AFTER the strings given won't matter)

... AND... immediately after the line given a line mostly of digits, but with "+" or "-" at the locations indicated by "s" in the previous line, e.g....

hhhhsttthhhhstttb 0471-0230262+2480

That second line will be dynamic. I.e. it will change from time to time.

It contains the data the Arduserver is... umm... serving. (Hence the name!)

In the example given, FW003 would break the line down as follows...

0471: First parameter, Param "0". A simple integer. Probably a count of something. Turns of an anemometer? Drops of rain? Anything the person who set up the Arduserver wishes!

-023: Second parameter. This will be "converted" by FW003 into -0.23. The Arduservers I've set up to date put a temperature into this parameter in deci degrees Celsius. So "-023" is taken to mean 0.23 degrees Celsius below zero.

0262: Third parameter. Another simple integer. Again, probably a count of something. But something else that what the first parameter reported.

+248: Fourth parameter. This will be "converted" by FW003 into +2.48, probably another temperature in deci degrees Celsius, but again: Whatever the person who set up the Arduserver wished to have reported here.

0: Fifth and last parameter: At present, FW003 doesn't use this, but you can see what the value of it is at the moment, and in the recent past, if you are looking at a running instance of FW003. Eventually, this datum will also be put on the graph that FW003 draws, and recorded in the data log it records.

===

Whew! That's a lot... but those two lines from the webserver are at the heart of everything FW003 is for.

So... you've got access across LAN or web to a suitable webpage.

((TKB's))

Look at the page's source. (Firefox: Right click on the page, request "View Page Source". Make a note of what line the "hhhhsttthhhhstttb", counting things as a non-geek would. If the line is on the first line, call it "line 1". Put that number aside. You'll use it in a moment.

===
Set up a folder for FW003. (You can have multiple folders for multiple copies of FW003 and its associated files, if you anticipate monitoring more than one webpage.)

Fetch the .zip file I provide, and unpack from it to the folder you created at least FW003.exe and FW003ini.txt.

Open FW003ini.txt in a text editor. Find the line...

WhereFmtCodeLineIs=

.. and put the number of the line with hhhhsttthhhhstttb there.

The line...

CodeForTypeOfDataSource=6

...should stay set to 6. (Some lines don't matter! (e.g. the items in the [Times] section.))

The section...

[OtherParamSettings]
DataTypeCode 0=0
DataTypeCode 1=1
DataTypeCode 2=0
DataTypeCode 3=1

... should also be left as it is.

The line...

IPAddr=

... should be set to the base url of the webpage you want. If it is served on a non-standard port, add a colon and the port number, e.g....

mon277rr.dyndns.org:1034

... for one of my Arduservers which is usually online.

Next....

Page=index.htm

is probably what you want. If the webpage isn't the default page of the site, change "index.htm" to the page you want.

The [WindowPosnAndSize] section is where the ini file saves the position and size of FW003's three windows each time you close the program, so that they open the next time where you last had them.

The section...

[PointPlotting]
Offset 0=1
Offset 1=50....
;
ScaleFactorx1000 0=8
ScaleFactorx1000 1=1000...

... can give you grief. You only need Offset and ScaleFactorx1000 entries up to the number of parameters you are following. At the moment, that would be up to Offset 3 (the FOURTH one) and ScaleFactorx1000 3.

Note the space between the name and the index number... "Offset(space)0" is right, "Offset0" won't be recognized. Ditto for the other.

The offset setting lets you "slide" lines up and down the graph screen, if they are appearing in an inconvenient place. Everything "wraps", so a "bad" offset doesn't matter much

The "ScaleFactorx1000" item is a bit trickier. If you set it to 1000, if the datum you are plotting increases by 1, the point plotted is one pixel lower on the screen. A ScaleFactorx1000 of 1000 is appropriate for the temperature data. (A TENTH degree Celsius will move the point one pixel, by the way. The plotting routines "see" the number as presented by the web page.)

If you are, say, watching an anemometer, where the number changes by 20 or 30 or more counts between readings, the slope of the line would become impossibly steep, and you wouldn't see a line. You'd just see a "spray" of apparently random dots, as the line wrapped (vertically) and wrapped, and wrapped again. If you set ScaleFactorx1000 to a number BELOW 1000, that will "squash" the data on that line. Set it to 100, and it takes a change of 10 in the reported number to move the dot plotted by 1 place. Set it to 10, and it takes a change of 100 to move the dot 1 place.

Getting the ScaleFactorx1000 right for your webserver is something of a trial and error business. And therein lies a "gotcha"....

You can leave the .ini file open in your text editor the whole time... but each time you quit FW003, the ini file is re-saved... with the values then present in the application.

Quit FW003 BEFORE making changes to what you see in the ini file. THEN (re)save the ini file from the text editor. That way, when you next start FW003, what it sees in the ini file will be in step with what is on the screen in the text editor.

Moving on....

The [Colors] is nice and simple, and you don't NEED to make changes here. It determines the color of the lines on the graph. If you say...

PColor 0=FF0000

... then the first parameter's line will be a pure, and as dark as possible red.

The "code" ("FF0000") works like this... it is in three parts...

rrggbb

For any position, "00" means "none of that", "FF" means "as much of that as you can".

The first two characters determine how much red there is in the color, the next two how much green, and the last two how much blue.

"80" means "half of max", "40" is 1/4 max, "C0" 3/4 max. So...

"40FF80"

...would be a color with a little red, max green, and "half" blue.

"000000" gives black. "FFFFFF" gives white... a poor choice, as the "paper" is white, and a white line on white paper is hard to see!

That's all on the ini file!

Be careful that you don't ask your computer to start two instances of the SAME FW003 running concurrently, by the way.

It is okay to start the program twice if you have two separate folders with separate copies of the .exe and .ini file. But the program writes to a data file (DtLgFW003.txt) in the same folder as the .exe which is trying to write. And two instances trying to write to the same .txt file doesn't work at all well!

Where have we got to?

So... you have, at least in theory, access to a webpage serving up the data in the right format.

You've got FW003.exe in a folder of its own, with a tweaked .ini file. (The main things that HAVE to be changed are the IPAddr= and (maybe) WhereFmtCodeLineIs= lines. Most errors elsewhere may lead to poor graphs, but won't be deal breakers.

Now you can run FW003!

You anti-malware software may need to be told to let the program "do" TCP to and from the Arduserver. It probably SHOULD ask if the activity is sanctioned by you!

Often there's a few seconds... say up to 10... while "things" settle down, but before long, you should start to see lines appearing and disappearing in the panel I've tinted blue in the screenshot to the left.

That panel shows you the data which is being recorded to the data file. (It is recorded in batches, hence the filling and emptying of the panel. This saves your hard drive from "thrash". Any un-saved data is passed to the data file as the program shuts down, by the way.)

I'll explain how to interpret the data in the data file later.

The "FW003_readings_f1" window gives you a look at the raw data lines, as extracted from the webpage.

And the third window has a graph of the values returned by the first four parameters. (The fifth, if your data source supplies it, isn't ignored for any complicated reason... I just haven't got to doing the necessary programming for that yet!)

On the graph, by the way, as the numbers coming from the server INCREASE, the lines go DOWN. (Again, a bit of programming to be done. Sorry! The program IS free!)

You will see some vertical lines on the graph. The magenta ones are done at midnight... on the machine that FW003 is running on. The gray ones are done at 6am, noon, and 9pm.

Interpreting the data file

While FW003 is running, it records the data it fetches from the webpage in a text file that has been designed to make accessing it for further analysis easy. The data file is called DtLgFW003.txt, and is recorded in the folder the .exe that creates it is in.

The data file can, for instance, be imported into any sensible spreadsheet program.

The file is created if it doesn't already exist. If it does exist, new data are added at the bottom, i.e. they are appended.

When FW003.exe is not running, you can simply rename the current DtLgFW003.txt if you wish to "set aside" the old data, and start a new file.

Here's a snippet from a data file on my machine....

17112v38,0, 0410,    6
17112v38,1, +241,    6
17112v38,2, 0012,    6
17112v38,3, +096,    6
17112v39,0, 0416,    9
17112v39,1, +241,    9
17112v39,2, 0012,    9
17112v39,3, +096,    9
17112v39,0, 0422,   12
17112v39,1, +241,   12
17112v39,2, 0012,   12
17112v39,3, +096,   12

.. and this is how you read it:

First column: Date and time datum was recorded. (More in a moment)

Second: The parameter number. (At 1/17, FW003 is hard coded to record FOUR parameters to the data file, parameters "0". "1", "2" and "3". From notes elsewhere here, I hope you recall that parameters 0 and 1 are counts of something wired to the Arduserver (or other data source) and parameters 1 and 3 are temperatures in deci degrees Celsius. (E.g., +241 in the data file stands for 24.1 degrees celsius above zero.)

Third column: the reading on that parameter at this time.

Fourth: The "cycles count" from the Arduserver (or other data source) at the time the reading was taken. (This is mostly useful for flagging times when the server was for some reason restarted.)

Date/time encoding

You're going to gag at the following, I bet... but it really does work well, when you are used to it. I've been using it for many, MANY years, in multiple contexts.

In the example, the first datum was recorded at "17112v38"... which is...

January 12th, 2017, 22:38 (10:38pm)

That is extracted from 17112v38 as follows...

yymddhmm

First two digits... the year in 20xx

Next character: Month. 1-9 for Jan to Sept, Oct: a, Nov: b, Dec:c

Next two: Day of month

Then a letter... for the hour in the day... "a" is first hour after midnight, "b" is 1:xx (am), "c" is 2:xx (am), etc.

... and lastly, two more digits, the minutes past the hour.

Compact. And if you do a simple sort of records, on this column, they will end up in chronological order.

And you can easily "break out" the information into more traditional forms, should you wish to do so.

"Quis custodiet ipsos custodes?"? The link will take you to the Wikipedia entry.

Ad from page's editor: Yes.. I do enjoy compiling these things for you... hope they are helpful. However.. this doesn't pay my bills!!! If you find this stuff useful, (and you run an MS-DOS or Windows PC) please visit my freeware and shareware page, download something, and circulate it for me? At least (please) send an 'I liked the parallel port use page, and I'm from (country/ state)' email? (No... I don't do spam) Links on your page to this page would also be appreciated!

Other projects