02/02/2016

Nova Launcher for Android

I finally got round to addressing the fact that on my Motorola Android phone,  I never used the Google search bar on the home screen. This was preventing me from having a whole row of app shortcuts.

The standard Motorola launcher does not allow this search bar to be removed – articles online suggest disabling the Google Search app, but this was not displayed as an installed app on the Motorola.

The Nova Launcher app on the play store was fantastically easy to install, and allowed me to remove the pesky search bar from the home screen.

Job done.

23/01/2016

Laravel vs. AngularJS

I have been spending some time keeping my PHP skills sharp, and have been dabbling with rewriting/updating a front end for my personal home-hosted websites.

I started off with Laravel, and was pleasantly surprised with how quick and easy it was to get off up and running  (manipulating data with CRUD), thanks to the excellent docs, and my forays into PHP MVC stuff (the yii framework) in the past.

Part way through however, I chanced upon this youtube video, and such was the simplicity demonstrated in the examples, I was inspired to go knock together a quick proof of concept.

After one PHP script (less than 50 lines) to spit out all the data I wanted as json, I had a tabulated data viewer with powerful live filtering.

Next on my list is to evaluate how easy it is to implement the search/filtering with the Laravel side, and compare my options for implementing a CRUD system on the single page AngularJS site.

An interesting exercise – I don’t feel like my time has been wasted by doubling up on the work.

15/01/2016

Creating Word documents programmatically

I have spent much of this week creating software to generate electrical certification documents in word based on data filled in on a mobile application in the field.

I must say that the Novacode DocX library has made the whole thing quick and painless, and what’s more, the whole thing is free to use.

Great stuff.

08/01/2016

Marketing yourself as a developer

I was listening to this podcast last night on Software Engineering radio, and there were a couple of points that really hit home – particularly at this time of year where traditionally resolutions are made:

  • Developers typically do not like to market themselves or their ideas.
    • A developer can write the best code in the world, but if nobody knows about it then it is all a bit pointless – no users will ever benefit.
    • Demonstrating the communication skills and commitment to document projects you are working on is highly attractive to prospective clients and/or employers, and could also earn you a following with among your peers.
  • Developers engage in traditional ‘Networking’ only in short and sustained bursts, usually at the time they are looking for a new job.
    • This needs to be an ongoing process, such that a strong network of connections is built and maintained.

The key to both of these is consistent commitment.

It only takes 10 minutes to write a blog post about something interesting done over the past week.

It is easy to take a few minutes to chat with member of your professional network.

I for one will try my best to put these principles into practice throughout 2016.

 

 

05/01/2016

Force a favicon onto a webpage – 2016 update

Back in 2011 I wrote this article about supporting multiple sites on shared hosting, and forcing each site to have it’s own favicon.

Modern browsers require image files, and it is just older browsers (and IE) that require the .ico files:

<link rel="icon" type="image/png" href="http://yourdomain.com/favicon.png">
<!--[if IE]><link rel="shortcut icon"href="http://yourdomain.com/favicon.ico"/><![endif]-->
24/07/2015

Making the jump to Linux

Switching from Windows to Linux for my main home computer (laptop) has been a real possibility for me for some time now, and I have finally made the jump – and I’m so glad I did!

All that my main computer gets used for is internet browsing based stuff, light word processing tasks, some web development and managing my (extensive) audiobook collection.

The only things left to tie me to Windows were a familiarity built up over my entire adult life, a natural apprehension towards change, and the fact that I use Windows day-in-day-out at work.

The final push I needed was the Microsoft announcement that .NET web applications (what I develop at work) are going open source, and are to be released officially for Linux.

I have been very surprised as to how easy the transition has been.

I chose the Lubuntu 64 bit OS, because my laptop is old. It had become really slow under the reign of Windows 7, even after Windows reinstall. It is

Here are the apps I have installed:

I have rewritten into Python some bespoke apps I had written in C# to auto-id3-tag and add to a database the radio content I record using get_iplayer tool. I may do a future article on this.

I have also followed the outstanding instructions here to get the basic Microsoft MVC5 ‘Hello World’ web app deployed.

The bottom line, Linux is great – and I won’t be going back Microsoft in my home setting.

26/01/2014

Automatically generating playlists (.m3u) from directory structure

I have many audiobooks that I like to listen to. With my growing collection, I thought it would be a good idea to set up some streaming on my internal network via a browser.

To cut a long story short, streaming is easy one file at a time – open the appropriate folder on the server to allow a web browser to browse to it. When that file is finished, click back and go to the next. Obviously this is OK for mp3 files of 30 minute episodes, but not so hot on a folder containing 40 x 5 minute tracks.

A playlist enables all the mp3 files to be played in a particular folder without any intervention. The pertinent C# code I have as a working starting point is as follows:

public void ProcessDir(string sourceDir)
{
StringBuilder sb = new StringBuilder();
StringBuilder sbPlayList = new StringBuilder();
List ls = new List();

string title = "";
string playListTitle = "";
string oldPlayListTitle = "";
int ct = 0;

foreach (string fileName in Directory.EnumerateFiles(sourceDir, "*.mp3", SearchOption.AllDirectories).OrderBy(filename => filename))
{
title = Path.GetDirectoryName(fileName).Replace(sourceDir,"");

string[] words = title.Split('\\');
if (words.Count() > 2)
playListTitle = words[2]; // the title eg. \author\title
else
playListTitle = words[1]; // just the authors name - no title subfolders

title = title.Replace(" ","%20");
title = title.Replace("\\", "/");
if (!ls.Contains(title))
{
ls.Add(title);
if (ct > 0) // don't write an empty playlist on the first one
{
sb.AppendLine("Adding " + oldPlayListTitle + ".m3u");
StreamWriter playListFile = new StreamWriter(cbDestination.Text + oldPlayListTitle + ".m3u");
playListFile.Write(sbPlayList.ToString());
playListFile.Close();
//sb.AppendLine("Contents of sbPlayList are: " + sbPlayList.ToString());
sbPlayList.Clear();
}
sb.AppendLine(playListTitle.ToUpper());
playListTitle = playListTitle.Replace(" ", "_");
oldPlayListTitle = playListTitle.ToLower();
ct++;
}
sb.AppendLine(cbPrefix.Text
+ title + "/"
+ Path.GetFileName(fileName).Replace(" ", "%20"));

sbPlayList.AppendLine(cbPrefix.Text
+ title + "/"
+ Path.GetFileName(fileName).Replace(" ", "%20"));
}
tb.Text = sb.ToString();
}
}

I am using a VLC media plugin with Chrome to play the m3u playlists directly from a link.

With my Android phone it is slightly more complex:

  • In the Dolphin web browser click the link to the m3u.
  • Choose to download the file with ES File explorer
  • Finally open the playlist with the Just Playlists application.

I make no apologies for the brevity of the explanation – still busy testing the above. I will do a more thorough write up when I have finalised my solution.

06/11/2013

BBC iPlayer alerter – Part 3: Email

Before reading this article you should read the previous two posts of this series: BBC iPlayer alerter – Part 1: JSON processing, and BBC iPlayer alerter – Part 2: Data storage.

The final thing I had to prove was that I could send an email from a Python script. A very trivial task as it turned out:

#!/usr/bin/python

import smtplib

sender = 'sender@address.co.uk'
receivers = ['recipient@address.co.uk']

message = """From: From sender <sender@address.co.uk>
To: To recipient <recipient@address.co.uk>
Subject: SMTP e-mail test

Email text ...
"""

try:
smtpObj = smtplib.SMTP('mail.address.co.uk', 25)
smtpObj.login("smtpuser", "smtppwd" )
smtpObj.sendmail(sender, receivers, message)
print ("Successfully sent email")
except SMTPException:
print ("Error: unable to send email")

Bringing all 3 elements together was less trivial – in essence I create arrays of RadioShow objects and compare those. Hopefully the comments in the code will explain most of the rest (please note – the WordPress <code> tags have messed up the indentation of my Python, which, if you know Python, is a big deal – I’ll try and fix this at a later date):

#!/usr/bin/python

from urllib import request
import json
import pymysql
import datetime
import smtplib
import re

# get the current shows as json feed
url = "http://www.bbc.co.uk/radio4extra/programmes/genres/drama/current.json"
response = request.urlopen(url)

encoding = response.headers.get_content_charset()
json_object = json.loads(response.read().decode('utf-8'))

class RadioShow:
def __init__(self, id, title, short_synopsis, current):
self.id = id
self.title = title
self.short_synopsis = short_synopsis
self.current = current

#create an array to hold the current shows from the json
radio_shows = []
#create an array to hold the existing shows from the database
existing_radio_shows = []
#new shows - add these to the email alert, and also to the database
new_radio_shows = []
#old shows - update these in the database so that current = n
old_radio_shows = []

current_date = datetime.datetime.now().strftime("%Y-%m-%d")

programmes = json_object['category_slice']['programmes']

i = 0

# read the current shows into the programmes array
for program in programmes:
#print (program['title'])
#print (program['short_synopsis'])
radio_shows.append(RadioShow(i,program['title'],program['short_synopsis'], 'Y'))
i += i

if len(radio_shows) == 0:
print("ERROR: no shows found from the BBC website")
exit()

#for item in radio_shows:
#print(item.title)
#print(item.short_synopsis)

# connect to the mysql database

conn = pymysql.connect(host='127.0.0.1', unix_socket='/var/run/mysqld/mysqld.sock', user='username', passwd='pwd', database='radioshows')

cur = conn.cursor()

cur.execute("SELECT id,title,short_synopsis,current,detecteddt FROM radio_4_extra WHERE current='Y'")

for row in cur:
existing_radio_shows.append(RadioShow(row[0],row[1],row[2],row[3]))

#loop through the radio shows from the json to check for any new ones
for item in radio_shows:
#print("first for loop")
exists = False
for existingitem in existing_radio_shows:
#print(item.title + " " + existingitem.title)
if item.title == existingitem.title:
exists = True
break
if exists == False:
# insert into database, and into the list for emailing
#print("Execute insert")
cur.execute("INSERT INTO radio_4_extra (title,short_synopsis,current,detecteddt) VALUES ('%s','%s','Y','%s')" % (re.escape(item.title),re.escape(item.short_synopsis),current_date))
new_radio_shows.append(RadioShow(1,item.title,item.short_synopsis, 'Y'))
conn.commit()

#loop through the radio shows from the database to check if any have gone from the json
for existingitem in existing_radio_shows:
#print("second for loop")
exists = False
for item in radio_shows:
if existingitem.title == item.title:
exists = True
break
if exists == False:
# its not there anymore - update the existing item so current=N
#print("Execute update")
cur.execute("UPDATE radio_4_extra SET current='N' WHERE id=%d" % (existingitem.id))
old_radio_shows.append(RadioShow(1,item.title,item.short_synopsis, 'Y'))
conn.commit()

cur.close()
conn.close()

# send an email
sender = 'sender@address.co.uk'
receivers = ['recipient@address.co.uk']

message = """From: From sender <sender@address.co.uk>
To: To recipient<recipient@address.co.uk>
Subject: Radio Show Alert

New shows detected ...

"""

for item in new_radio_shows:
message += item.title
message += "\n"
message += item.short_synopsis
message += "\n\n"

try:
smtpObj = smtplib.SMTP('mail.address.co.uk', 25)
smtpObj.login("smtpuser", "smtppwd" )
smtpObj.sendmail(sender, receivers, message)
#print ("Successfully sent email")
except SMTPException:
exit()
#print ("Error: unable to send email")

06/11/2013

BBC iPlayer alerter – Part 2: Data storage

Before reading this article you should read my first post of this series: BBC iPlayer alerter – Part 1: JSON processing before you dive into this one.

You will remember that for step 2 of my plan to automate detection of new BBC iPlayer shows, I had to compare the shows detected in the current run through the JSON. This obviously means that all the shows that were on last week had to be stored somewhere, so they could be compared for changes (new show would not have been there last week; also shows that are no longer current will not be there).

The obvious place to store all the information is within a database, and, as the application is sitting on a Raspberry Pi, MySQL is the best choice.

Predictably, I wrote a test program for reading from and writing to a MySQL database from Python. For this, I used the pymysql library, which is excellent:

#!/usr/bin/python

import pymysql

conn = pymysql.connect(host='127.0.0.1', unix_socket='/var/run/mysqld/mysqld.sock', user='username', passwd='pwd', database='your_database')

cur = conn.cursor()

cur.execute("SELECT * FROM your_table")

for row in cur:
print(row[1])

cur.close()
conn.close()

What took me longest to get this working is the socket in the pymysql.connect() function. The usual port (3306) did not work, for reasons not entirely clear to me. In the end I had to put in unix_socket=’/var/run/mysqld/mysqld.sock’, as you can see above.

In a similar script for testing the insert, it took me a while to realise conn.commit() had to be called for it to work.

06/11/2013

BBC iPlayer alerter – Part 1: JSON processing

As I am unable to read during my morning commute, I have taken to listening to audiobooks to fill the time usefully.

A great source of material to listen to are the shows on BBC Radio 4 Extra. I have a technique for getting the iPlayer content onto my generic mp3 player, but I found trawling through the lists of upcoming programmes to earmark those I wanted to watch rather laborious. 

To automate this process, I wanted to get an email on a weekly basis telling me of new upcoming shows. I would then just need to read through the handful of new shows that had been added, as opposed to the entire list. The only computer that I keep on all the time is my Raspberry Pi, so I decided a Python script run using a weekly cron job would meet the case.

I split the problem into 3 basic steps (hence splitting the posts for the project into 3 parts):

  1. Automatically read all the current shows
  2. Compare the list of current shows to those of last week
  3. Send an email of the newly detected shows to me

To accomplish point number 1, I discovered that the main list of shows I browsed regularly (Drama) was available in JSON format by appending .json to the url: http://www.bbc.co.uk/radio4extra/programmes/genres/drama/current.json

Resulting in the raw information about all the current shows. A small sample is below:

{"type":"series","pid":"b01pvbbs","title":"David Constantine - Tea at the Midland","short_synopsis":"Short stories from one of the London Evening Standard's Books of the Year 2012.","image":{"filename":"tea-at-the-midland_midland-hotel-morecambe_140113_s_get.jpg"},"is_available":true},{"type":"series","pid":"b00yjr30","title":"Dick Francis - Dead on Red","short_synopsis":"Simmering resentment brings a high class hit-man over from France to target a jockey","image":{"filename":"dick_francins_dead_on_red_0211_s_get.jpg"},"is_available":true},{"type":"brand","pid":"b01ms5v9","title":"Dickens Confidential","short_synopsis":"Before writing his novels, Charles Dickens worked as editor on a newspaper...","is_available":true}

The next thing to do was to process this using a Python script.

Here is the first test:

#!/usr/bin/python

from urllib import request
import json
url = "http://www.bbc.co.uk/radio4extra/programmes/genres/drama/current.json"
response = request.urlopen(url)

encoding = response.headers.get_content_charset()
json_object = json.loads(response.read().decode('utf-8'))

programmes = json_object['category_slice']['programmes']

for program in programmes:
print (program['title'])
print (program['short_synopsis'])

At this stage, the titles and details of the radio shows are printed on the screen.

I will tackle the use and storage of this data in the next post.