Touch Typing Week #3

Time for another update on my touch typing skill progress: after 3 weeks I can say that I definitely see improvements, the chart will tell you the same, however I am not ready to touch-type yet (tried to type this post using this method but it was so slow and frustrating that I had to do it the old way :) )

The main problem now is that the tool does not use any other keys except letters and when I need to press “backspace”, correct a mistake or place quotes – this just ruins the flow completely and slows down things tremendously – I wonder if the tool has some exercises for those keys too or I will have to just adapt to other keys on my own.

Anyway besides me being lazy and skipping some days of practice it is working and I am going to continue, the initial estimation is surely going to be blown off by a significant margin but hey, I still want this skill – probably couple more weeks and English will be conquered, then it’s Russian.

touch-type-week-3

SProt Client Mode State Diagram

Client side of sprot is finally ready too. Well, at least main logic is defined. While I was at it I understood that sequence numbers of data frames will be needed to resolve situations when server accepted a frame X and sent an ack but client did not receive this ack from server  for whatever reason and retransmitted the same X frame. In a such situation the server should just ignore the frame and answer with ack again.

Introduction of the sequence numbers caused some changes on diagrams posted previously, so those posts were updated. The sequence number will be reset each time mode switch frame is received and mode switch is performed, also because it’s only a single byte sequence number will be reset to 0 after reaching value of 255.
sprot client states

SProt Server Mode State Diagram

Here is how message exchange looks like from sprot server perspective. Diagram could be optimized by having a generic frame received state followed by crc check and sending nack in case of failure, but I realized it too late – already spent more than hour drawing that thing so I just left it “as is”.
sprot server states

Meet SProt, FPLog Communication Protocol

As one of the features of FPLog should be an ability to send arbitrary large log messages, communication protocol with packet assembly is required. The same protocol will be used for application to fplogd and fplogd to log collecting server communications. For now I am thinking about having only blocking IO but it might change later. Also this simple protocol (or just “SProt”) might be moved to a separate library. Maybe I am reinventing a wheel a bit with this protocol but in this project I do have such luxury because the goal is not to make it happen as fast as I can but to gain knowledge and practice my craft to become better, I have enough of reusing stuff at work so at least here I can make dumb choices from time to time :)

Here is an overview of the frames protocol consists of, state diagrams and logic will be described in later posts.

sprot frames*CRC7 polynomial 0×91 was used for CRC calculations.

FPLog Design Overview

Spent several hours thinking, selecting the drawing tool and finally making the sketch. Decided to try some diagramming tools available for Android tablets and found one that I liked very much (DrawExpress Diagram) – it allowed me to produce quality schematics pretty fast, learning curve is very merciful and the tool supports various connector types to show OOP relations, “bring to front” / “send to back” functions to create composite objects like boxes inside boxes, etc. Of course there is a group selection ability and finally the most interesting part – the tool is gesture-based, there is a number of simple to remember gestures that draw stuff on screen.

However not everything was as smooth as I wanted it to be: when I was making final strokes to fplog draft design the application crashed and it started to crash every time I tried opening fplog.de file.. I thought that I would have to start over and was quite disappointed as you may imagine but there was a possibility the format of the file is human-readable like XML or something and there is a chance I can fix the corrupted file to make DrawExpress work with this file again.

Docked my Galaxy Note 10.1, fired a terminal, switched to root to see all files (yes, my tablet is rooted) and executed

find / -type f -name "*fplog*"

That gave me a file inside “/data/data/com.drawexpress.full/files/” so I copied it to a non-root accessible folder, chmod to 777 and copied to my PC over FTP.
When I opened fplog.de file I was pleased to find simple JSON inside, it looked like there were all objects sorted by creation time followed by array of connectors that connected all these objects. When the editor crashed I was trying to add couple of connectors, so I decided to remove several connectors from the end of the file. Well, few JSON objects later I was overwriting my original fplog.de file on my tablet with the one edited on PC.
Rejoice! It worked and DrawExpress stopped crashing, here is the design I was so eager to save:

fplog_design_draft

Touch Typing Week #1

Well, I must say that it is harder than expected and also I was too lazy to do it every day at least not for 30 minutes (on average I was touch-typing for 21 minutes a day). Anyway there is some progress, see the chart below (don’t pay much attention to the first 2 days – I was not positioning my hands correctly for doing proper touch typing, on the 3rd day that was fixed and speed dropped almost below zero :) along with the number of typos). So now I am able to touch-type 12 words per minute doing 10 errors, that’s it for the first week.

touch_type_week1

FPLog Begins

In my previous post on logging I wrote about how it could be future-proofed and now I would like to give this concept some development.. literally – I want to develop a logging solution based on the concept presented earlier.

The solution will consist of client and server parts.
Client:

  • C++ and Java API
  • Various transports (Ethernet, Bluetooth, Serial, USB)
  • Windows, Linux and Android support

Server:

  • MongoDB as persistent storage
  • GUI application in Java and Spring
  • Windows and Linux support

With this project I would like to practice multiplatform development, refresh my rusty Java in the process and finally create a logging solution that I can use later in some other projects. Well, let’s see how it goes, hopefully not like 99.999% of such hobby side projects (hint: dev/null :) )

By the way the name of the project is ‘fplog’ which reflects the idea of this solution being future-proof.

Future-proof Logging

Some time ago I had a project that required working with lots of exchange orders, these orders were arriving into the system in hundreds per second: some were filled, others modified or canceled. In other words there was always some processing involved that needed to leave some traces in order to understand what happened with the particular order in case of some client dispute.

Project was developed from scratch therefore it was my decision on how to introduce logging there. As the feature seemed low priority not much time was invested in it and as a result a simple unstructured text logging with timestamp & priority was implemented. It was very human-readable and seemed pretty good until the project went public…

Every week I got a few gigabytes of text logs to process in order to find this or that problem or just to make sure the system is functioning as designed and everything is fine. During that time there were couple of severe problems detected very late because of this very inconvenient and time-consuming process.
After thinking over this mess several things became clear: text files are very inconvenient logs storage and a Nagios probe (or some other automated solution) should be reading the log in my place.

Logging mechanism improvements were in order: introduce some structure to log messages (make some mandatory and optional parameters that every log message has) and use database for logs storage.
After implementing these improvements we could easily search DB for errors ourselves or setup some Nagios probes that regularly do it for us.

To be fair I must confess that I left the company before improved logging mechanism was introduced, it was only half-way there, meaning that only some of the log messages were stored to DB – only those messages that were a consequence of some exception happening in the code and automatic probes could analyze only this portion of log searching the DB for the red flags so to say.

Anyway even this portion of the log stored inside the DB showed that some problems still remained with this approach: SQL database with its fixed schema was too rigid and inflexible storage for the log messages which can have arbitrary number of arbitrary parameters.
The answer is to store the logs into a NoSQL schema-less document database in an easily-parsable form which allows any parameters for a specific message.

I should give a guy who wrote this post a credit for making a case of using JSON as log message format, he also mentions systems that can be used for storing, searching and presenting the logged information.

All of the above makes me think that using the structured logging and storing everything to schema-less DB provides a pretty flexible solution that could withstand a test of time and in particular the everchanging requirements which is the hardest test for any system. Even the syslog protocol has some place for structured log information since RFC 5424 (see 6.3. STRUCTURED-DATA section) although this RFC is still in a proposal state.

Learning Touch Typing

The first skill I would like to master in 2014 is touch typing. I made several attempts previously but failed because the task is too boring and repetitive.. Some say the skill is essential for programmers some disagree and there seem to be no real facts – only opinions. Well, I am going to test it either way because despite the typing speed is not so important while coding, thinking about typing distracts from the code and ideas that you are trying to code.

I intend to train for 30 minutes a day and report the progress. Searching the net for touch type tutors I have found one solution particularly popular in Russian segment of the net, called ErgoSolo. Downloaded, installed, tried, puked, uninstalled. I will not even give you a link to this crapware – wants to install a bunch of Yandex shit together with it and also has a very unfriendly DRM – install only on 1 PC, if there are hardware changes to your PC you need another registration code and if hardware changes occurred after 6 months from purchase you will have to repurchase, no kidding..

Long story short, I am using keybr.com, so far so good – supports languages I need (English and Russian) and is free for now at least. Currently I am touch-typing like a pro with astonishing speed of 12 words per minute making mere 22 mistakes ;) Hopefully this situation improves in the coming weeks (target is 5 weeks for both languages, digits, functional and other keys).