Fix Skipping Videos Running OSMC on Raspberry Pi 3

Issue: no matter the network connection speed videos which are larger than ~12GB per 1.5 hours are constantly skipping, buffering and are completely unwatchable.

The proposed fix will work only if your network bandwidth really could accommodate the speed needed for seamless video streaming. This is easy to check – say you want to see if your 20GB video should in theory play smoothly on the network player. In order to check this you could start copy process from your network storage to your workstation via Samba or FTP (or anything else you can think of) and see the estimated time of file transfer. Let’s imagine it says that your 20GB file will be copied in 50 minutes, in case the video playtime is 120 minutes you can be sure that it should be possible to stream this file seamlessly to a network player and you should not experience any stutter. In reality I am facing the issue described at the very beginning of this post.

Fix: apply fix described here in Example #4.

SSH to RPi

And check KODI version, for fix to be simple copy-paste the version should be >= 17. Currently NOOBS installation of OSMC is supplied with 17.6 by default. In case your version is older, please refer to the linked fix and read how settings names should be changed for older versions. Obviously you have to know the IP address of your RPi device with OSMC installed, it is easy to do either from System Info inside OSMC GUI or by checking connected devices on your router web interface.

Login: osmc
Password: osmc

Credentials are default, ssh is also on by default after OSMC installation.

Screen Shot 2018-04-15 at 18.38.05

Apply the Fix

Go to /home/osmc/.kodi/userdata/ and create file ‘advancedsettings.xml’ there. I have installed Midnight Commander for convenience and also used its embedded text editor for pasting the correct text inside the file.

Screen Shot 2018-04-15 at 18.39.33Screen Shot 2018-04-15 at 19.12.24
Now save the file and reboot by ‘sudo shutdown -r now’ from your ssh session. On the next boot you should have no more issues playing large video files.

Never cut power to your RPi without doing a proper shutdown from OSMC or other available OS GUI because it is easy as 1-2-3 to corrupt OS on your SD card and be forced to re-install the OS.

 

Shared Folders on Linux Guest OS in VMware (Fusion/Workstation/Player)

Having VMs running around you will eventually need to pass some files between your host and guest OSes. This could be achieved in a number of ways, one of which (and the most convenient long term in my opinion) is to have a shared folder between the host and the guest.

This writeup was compiled on Mac running Sierra thus VMware Fusion was used. However basically the same steps could be applied to Player or Workstation running on Windows as a host OS. Here is the version I used specifically:

Screen Shot 2017-07-07 at 20.57.11

0. You are working with VMware, not VirtualBox, keep that in mind

Believe it or not but because I was so used to VirtualBox, first of all I installed “VirtualBox Guest Additions” for Linux guest OS and was wondering why the damned thing does not work. Spent quite some time until it hit me that I was trying to load VirtualBox stuff on VM that is being run by VMware…

1. Make sure OpenVM packages are not installed

You would need tools from VMware in order to have shared folders functionality, I learned it the hard way so you don’t have to. OpenVM does not support file system for mounting VMware shared folders.

Make sure this package is not installed on your guest Linux OS:
Screen Shot 2017-07-07 at 21.01.09

2. Install VMware Tools

Select your VM from Virtual Machines Library and use VMware menu item called “Virtual Machine”, you will see “Install VMware Tools” or “Reinstall VMware Tools” there, click on it.
Screen Shot 2017-07-07 at 21.02.59
Please note that it has to be done while VM is running.

Almost immediately you should see VMware Tools CD image being mounted inside your guest Linux OS:
Screen Shot 2017-07-07 at 20.58.45

See the tar.gz archive there? You should copy it somewhere, for example to Downloads folder in your home dir and unpack. Once you’re done unpacking, please run vmware-install.pl as superuser.
Restart your guest OS.

3. Create a folder on host OS and share it with VM

You could create a folder anywhere you can write to on your host filesystem. Then go to VM settings->Sharing, enable folder sharing and browse to your newly created folder. To make sharing as painless as possible please refrain from using spaces in shared folder name.
Screen Shot 2017-07-07 at 20.56.23

4. Enjoy the fruits of your labour

/mnt/hgfs on your guest should contain the shared folder now:
Screen Shot 2017-07-07 at 21.05.03

Running CRON Jobs That Require GUI

I am using FreeFileSync utility for mirroring contents of several HDDs, call it software RAID if you must :) This software can be run in batch mode however unlike many Linux native utilities this one is not text mode by default and require more than just a terminal to run.
By any means this fact is not exceptional if you run FreeFileSync manually from some window manager, however things might not work as smoothly if you wish to run FreeFileSync at regular intervals with the help of cron for instance. That was exactly my case that I intend to describe here.

First of all I created a single script that should be called regularly by cron, the script contains all the FreeFileSync calls I need in one place. To test how the script works I added it to crontab and made it scheduled to be executed every minute:
crontab -e
cron_job_every_min
Now just to be on the safe side I restarted cron daemon:

cron_restart_and_status
Looking good, now inspecting the logs in /var/log/syslog:

cron_run_jobs

However I did not see any activity on FreeFileSync part and my disks were left unsynchronised.. What gives? What is important to know here is that cron sends you an e-mail in case it has some error to report but in order to get this mail you have to install mail server and some mail client. I went with one of the most popular choices: postfix for server with local-only configuration and mailutils package for text-based client which is called by executing mail command. Both postfix and mailutils should be available in your distro of choice, in case of Debian based flavours sudo apt-get install to the rescue!

Here is what I saw in the e-mail sent to me by cron:

mail_from_cron
It turns out the issue could be resolved with a simple trick – initialising DISPLAY variable inside the crontab itself. Here we need to remember that cron runs jobs with very limited environment and it is very common that jobs fail because of some environment variable is missing. Below I am setting the proper DISPLAY and make sure my command runs once a day instead of once per minute:

crontab_with_display
That was all I needed to successfully schedule my cron job. The takeaway is this: mind the environment working with cron, make sure to inspect syslog log and also have the local mail server properly setup in order to get mail from cron in case of issues running the jobs. By the way your inbox should be located in /var/spool/mail/ and the format of the file is human readable so maybe you won’t need mailutils, just postfix.

LAN Over Powerline

The idea is not new – I’ve been hearing about it for the last 10 years at least (check it out here). For about as long I am depending on WiFi to satisfy all my networking needs because apparently the electric wiring in my apartment was designed by an idiot with zero respect to high-speed networking… Anyway I decided to give it a shot and see if it fares better than WiFi.

TLDR: the answer is NO it does not.

I have started with the kit from Netgear, Powerline AV500.

20170407_181041643_iOS
Manufacturer specifically warns you:

Do not plug Powerline products into a power strip, extension cord, or surge protector.

Well, that was quite unfortunate because the same gifted individual that designed all the wiring was also very economical power-socket wise… Anyhow I’ve plugged adapters into spare sockets, securely paired them as per manual and fired a speedtest. WOW!! Whopping 12 Mb/s!!! Not bad for.. 500 GIGAbit adapters??! Well, at least they worked. Because they might not have worked at all – if my rooms were on different switch boards for example and not connected in any way – that’s it, no chance. Take that into account if you decide to try for yourself.

Unfortunately for me, that was the speed at the locations I required those adapters to be. Extension cords or no extension cords – situation was the same, I guess power grid in my neighbourhood is too noisy for such equipment to work optimally.

Fast forward several years and guess what? I’ve decided to see where the progress have taken this technology, enter Netgear Powerline 1200:

netgearav1200powerlineadapter-4900-001

Aaaand, Ladies & Gentlemen results are in!

Screen Shot 2017-04-07 at 21.29.16

An improvement? A slight one, yes, but the fact is – latest AC standard WiFi routers will beat the living shit out of these power-line adapters and that’s it for today folks.

 

Undetectable Ad Blocking With DNS and More

Morals are intentionally left out of this post, my focus is on the technology that allows to achieve the stated goal (block all ads) in the most efficient way.

Present issue with adblockers is that they are fairly easy to detect and many sites are doing that, detecting adblockers and redirecting viewers to some warning pages or payed subscription prompts – try www.forbes.com for example.

So, how can you achieve results that are better than say AdBlock Plus?

Use alternative DNS service

This approach will work for the whole network that is connected to the router, which DNS setting will be changed to the alternative DNS, not the one coming from your ISP. See the example of router settings below. Despite the fact that I get address from my ISP automatically I chose to set DNS server manually and 192.168.1.240 is the IP of my alternative DNS server that filters out advertisers’ domains and ads simply do not load. Bonus points: ads in ‘free’ mobile apps are also not showing on all devices, connected to the router. Also no other adblockers are required to be installed on your mobile devices/PCs/Macs.

Screen Shot 2017-04-02 at 18.45.20

Curious reader might wonder why the IP address of DNS server is within the local network, and the answer is Pi-hole. I have installed this app using steps provided on their site and should say it works like a charm out of the box! A bit more details if you’re interested which Pi model works with Pi-hole: I have installed latest Raspbian on Raspberry Pi 1 Model B and it seems to be powerful enough to run the service.

Screen Shot 2017-04-02 at 18.25.58

However you absolutely do not need to install anything or own Raspberry Pi device – there is a free service that filters away ads for you, see here and use the following IPs for DNS settings on your router: 198.101.242.72 and 23.253.163.53.

UPDATE: after several weeks of using the above mentioned DNS servers I suddenly got a message informing me about the end of trial period and if I want to continue using the service I would have to subscribe for 2.99$ a month. They do offer PayPal subscription which is convenient and more secure than just using credit cards.

AdBlock Plus with Anti-Adblock Killer

First of all you would need the gold standard of ad-blocking which is AdBlock Plus and now, to make it undetectable, follow the steps outlined here.
I was able to successfully install and use the combination on Firefox:
Screen Shot 2017-04-02 at 18.29.23
Unfortunately did not manage to get this stuff to work on Safari due to the Tampermonkey extension being unable to install for whatever reason.

Conclusion

Both approaches presented above could work independently, obviously DNS approach will not be available on networks, where you do not have access to router or you do not wish to setup your personal router.
Second approach will work in all networks and won’t require to make any modification to router settings, however it is less universal than the DNS filtering approach and some sites could still detect ad-blockers being used.

I am using a combination of the two which would also be my advice for those, who would like to get maximum efficiency in blocking the ads. It’s not guaranteed to work 100% of the time but it seems to be the best available method right now.

How to Boot with UEFI from USB Stick

Nowadays BIOS boot is considered to be “legacy” way of booting and EFI is the emergent default mode. However people still confuse the two and lots of tools for creating bootable media ignore EFI and go with standard BIOS boot.

So, what do you need to boot in (U)EFI mode?

  1. Obtain OS image/zip that is EFI-aware, having directory /boot/efi on the OS distribution is a good indication of EFI support
  2. Format USB stick with GUID(GPT) partition table and create a single FAT32 partition named using only CAPITAL letters
  3. Extract zip or image file into the root folder of your newly formatted USB stick
  4. Place your USB in non USB 3.0 slot, don’t know for sure why but you cannot boot from USB 3.0 slot

On OSX you can use Disk Utility for the above operations and on Linux, GParted comes to the rescue. I am sure you can easily do that on Windows too, either using standard admin tools or something like Partition Magic, however I did not check therefore can’t advise anything specific.

As for booting Windows and Linux in EFI there are guides how to do that, just search. Although the main requirement is GUID Partition Table, the best way of achieving that on your startup disk is to allow the Operating System to partition the drive during the installation as opposed to manual partitioning.

If you need to dual-boot into say Windows and Linux and you installed Linux in MBR mode but Windows in EFI then Grub will be unable to find Windows installation and you will have to select the correct boot drive on each boot, which will drive you mad eventually. Do not despair though, there is a great utility called Boot Repair, just load USB stick with some Live Ubuntu and then apt-get the boot-repair from there and you will have your situation resolved semi-automatically in the matter of minutes. Writing from personal experience as I was in exactly the above mentioned situation with dual-boot, this utility helped me to convert Linux loader into EFI and now it also allows me to select booting into Windows right from the boot menu.

 

DLNA Audio Player for iOS

KODI, how I wish you were available for iOS!

Well, technically if you are ready to jailbreak your phone or suffer through other ways of getting a development build on your iDevice, then KODI *is* actually available for iOS: http://kodi.wiki/view/HOW-TO:Install_Kodi_for_iOS#Development_and_custom_builds

However for the majority of people AppStore is where the apps are, so I started looking there, the obvious place, right?

2.5 hours and approximately 40 EUR later I sort of have found what I was looking for, had to install paid versions of multiple players just to learn that they are incapable of doing what I wanted them to do.. Am I just retarded or there is no easy way to ask for a refund on app from the AppStore, like on Android for example?

Oh yeah, if you mean this -> http://ioshacker.com/how-to/get-refund-app-purchased-app-store then FUCK THAT SHIT! I think I know now how AppStore gets much better earnings per app than PlayStore – it just makes it hard for people to get refunds for the crap they bought! Don’t buy crap you say? Well what if I need to test before I can give the verdict and there is no ‘lite’ version or ‘lite’ version do not have the feature I am after? Well, FUCK THAT SHIT AGAIN I AM PISSED!

With this rant out of the way I can get back to the core of the issue – what I was after, what the requirements have been.

Find a UPnP / DLNA player that can play folders served from the server and have ’shuffle play’ functionality.

That was it! I thought well, should be easy, basic stuff.. I do not wish to litter the post with swearing more but to put it mildly – expectation turned out to be very different from reality… Play individual tracks? Here you go, please, but playing the whole folder – NOOooo, soooo difficult!

Why did I write that I sort of have found the solution? The player I found is able to play folders but not the topmost folders, for example if you have a folder ’some shit’ and it has a subfolder ‘more shit’ then you will be able to play only the ‘more shit’ folder and only if that folder contains nothing but tracks, if it has tracks + some folders, you are screwed again. Did I mention that this player has UI resembling Pablo Picasso paintings more than human usable interface? The screenshot cannot show all the WTF? moments you are going to have, so if you are the first time user – good luck finding where to tap! Add to that the complete freezes where you have to kill the app and re-launch (thankfully do not happen while playing, rather happen when browsing through the DLNA server much).

IMG_0636

I am done with this search, Creation 5 will likely to stay with my iDevice – I have no mental power left to continue looking for alternatives and with this, still very much pissed, I would like to conclude the post.

Building Dependencies for fplog

Build Boost

Windows

First execute bootstrap.bat and then the following:
b2 --build-type=complete --toolset=msvc architecture=x86 address-model=32

Linux

g++-multilib: you need to install this package to do cross-compilation, building 32-bit executables on 64-bit machine.
Execute bootstrap.sh and then the following:

./b2 -j8 --build-type=complete --layout=versioned --toolset=gcc architecture=x86 address-model=32

-j8 switch here means that 8 parallel compilation threads will be used, rule of thumb should be number of CPU cores * 2. In my case I have four cores, each able to execute 2 threads in parallel, so my total is 8. Speeds-up compilation significantly.
Multi-core build, htop screenshot showing 100% load of all CPU cores.
After building for any platform please copy folders boost and stage inside your boost source tree into fplog/boost folder (you have to create this folder manually).

Building boost_python

Here is the build command:

./b2 -j8 --with-python --build-type=complete --layout=versioned --toolset=gcc architecture=x86 address-model=32

The output will give you boost_python libraries with your system default Python (most likely 2.7). In order to get boost_python3 libs you will have to make the following modification inside the project-config.jam file generated by Bootstrap.sh before executing the build (see the line with the cursor):

boost project-config.jam with python3 enabled screenshot

Build MongoDB legacy CXX driver

Prerequisite are SCons and Python 2.x.
Driver also depends on Boost, so Boost should be built before starting with MongoDB driver. Notice that I am telling scons to use Boost headers and libs that I built and put into fplog project.

Windows

scons install --32 --cpppath=c:\Users\kr0st\Source\fplog\boost --libpath=c:\Users\kr0st\Source\fplog\boost\stage\lib
scons install --32 --dbg=on --cpppath=c:\Users\kr0st\Source\fplog\boost --libpath=c:\Users\kr0st\Source\fplog\boost\stage\lib
scons install --32 --dynamic-windows --sharedclient --cpppath=c:\Users\kr0st\Source\fplog\boost --libpath=c:\Users\kr0st\Source\fplog\boost\stage\lib
scons install --32 --dynamic-windows --sharedclient --dbg=on --cpppath=c:\Users\kr0st\Source\fplog\boost --libpath=c:\Users\kr0st\Source\fplog\boost\stage\lib

Linux

scons install --32 --cpppath=/home/kr0st/source/fplog/boost --libpath=/home/kr0st/source/fplog/boost/stage/lib --boost-lib-search-suffixes=-gcc48-1_61,-gcc48-mt-1_61,-gcc48-mt-d-1_61,-gcc48-d-1_61,-gcc48-s-1_61,-gcc48-sd-1_61,-gcc48-mt-s-1_61,-gcc48-mt-sd-1_61

scons install –32 –sharedclient –cpppath=/home/kr0st/source/fplog/boost –libpath=/home/kr0st/source/fplog/boost/stage/lib –boost-lib-search-suffixes=-gcc48-1_61,-gcc48-mt-1_61,-gcc48-mt-d-1_61,-gcc48-d-1_61,-gcc48-s-1_61,-gcc48-sd-1_61,-gcc48-mt-s-1_61,-gcc48-mt-sd-1_61

scons install –32 –dbg=on –cpppath=/home/kr0st/source/fplog/boost –libpath=/home/kr0st/source/fplog/boost/stage/lib –boost-lib-search-suffixes=-gcc48-1_61,-gcc48-mt-1_61,-gcc48-mt-d-1_61,-gcc48-d-1_61,-gcc48-s-1_61,-gcc48-sd-1_61,-gcc48-mt-s-1_61,-gcc48-mt-sd-1_61

scons install –32 –sharedclient –dbg=on –cpppath=/home/kr0st/source/fplog/boost –libpath=/home/kr0st/source/fplog/boost/stage/lib –boost-lib-search-suffixes=-gcc48-1_61,-gcc48-mt-1_61,-gcc48-mt-d-1_61,-gcc48-d-1_61,-gcc48-s-1_61,-gcc48-sd-1_61,-gcc48-mt-s-1_61,-gcc48-mt-sd-1_61

Please note that because I am using versioned layout for building boost I have to use suffixes in order to compile mongo driver: –boost-lib-search-suffixes=-gcc48-1_61. Also take into consideration that suffixes depend on boost and gcc versions (here 1.61 and 4.8 accordingly).

Once build for any platform is done, copy build/install folder of MongoDB Legacy CXX driver source tree into fplog solution folder and then rename this install directory into mongo inside fplog.

You’re done!

NOTE: you have to use 64-bit version of MongoDB server, it is accessible from 32-bit application and does not have 2GB storage limitation.

From Plex to Serviio via MiniDLNA

I’ve decided to make a switch – NUC (previously wrote about it here) is replaced by Raspberry Pi 3 B in Media Player role and NUC itself will serve File Server role from now on. In order to do that, I have made some reconfiguration and also installed Plex on my NUC (which has Ubuntu 15.10 at the moment of writing this blog post). It did not go well..

Plex was visible from other devices but as soon as I try accessing files from DLNA-enabled device, Plex occupies all CPU cores almost at 100% and nothing happens, I mean content cannot be accessed. This problem did not go away despite my several attempts at fixing it. Finally I gave up on Plex and went searching for another options.

MiniDLNA was a logical next step as this is quite popular solution for embedded devices and Linux. Well, almost worked – music and videos were all right but collection of photos did not want to show up on Kodi or similar players on any platform. Randomly some photo would display after very long delay but it was so slow to show up to be considered unusable. MiniDLNA log file showed errors like this a lot: “Returning UPnPError 701: No such object error”. Somewhere on the forums I’ve seen people saying the error is insignificant and everything should still work. Unfortunately not my case – I was unable to fix pictures display on any of the media players when MiniDLNA was serving the content.

Serviio came up next in my search after the fiasco with MiniDLNA. It is Java application that runs from terminal by default (with an option to make it a daemon), it is configured by using a web interface (a-la Plex) and to my delight it actually handles all content pretty well, the problem is resolved, thanks, Serviio!

Fixing Screen Tearing on Linux Mint 17.3

I’m on Nvidia GF770 GTX, there is a decent, though proprietary, driver for my GPU from Nvidia but what bothers me a lot is that even despite having the correct driver with 3D-capabilities, able to run demanding 3D games I am still having a trivial issue of screen tearing in games and videos.. I have tried to enable V-Sync wherever I found the setting to no effect: still the same ugly tearing all over the video or a game scene.

After some research on the topic I decided to give Compton a try and because it has resolved the issue for me I stopped at this solution, here is the detailed guide on how to set this thing up: http://duncanlock.net/blog/2013/06/07/how-to-switch-to-compton-for-beautiful-tear-free-compositing-in-xfce/