A Control Panel/Dashboard

Here was my vanity project this summer:

For those of you with OCD who are twitching due to the misalinged bottom frames, this was fixed post screenshot. Sorry.

I’ve been meaning to do something like this for around a year – having an easy to view calendar not on our phones and controlling the MPD server above the kitchen were the main drivers.

I’m going to go through each component of the panel before giving an overview of how the whole thing hangs together, in an endeavour to kinda-sorta document the thing for when it inevitably breaks.

A Music Player (except not)

MPD is… quirky

I tried a few music servers before just settling on a raw MPD install. I was using a Raspberry Pi 3, but it kept freaking out with maintaining WiFi (no idea why) so I ended up with the current server – Ubuntu running on an old Mac Mini via a USB external HDD (since the SATA controller died on the Mac).

I’m astonished this thing works at all, to be honest. Even acts as a bluetooth speaker when needed.

The issue with the MPD server was in operating it – the MALP app remote controller is fine (though quirky in its own way), but not super convenient or transparent to use when wandering around the kitchen. What I wanted was an easily accessible control to simply play/stop/next and display track details.

There are existing web applications I could have used for this job, but they were either too hard to configure right or hugely overkill for what I wanted. None of them quite fit right inside an iFrame either, which was frustrating.

It’s not possible to create a pure JavaScript controller for MPD – it doesn’t allow for WebSocket connections, therefore any web app solution would need a CGI backend. To whit, fine, I’ll roll my own.

What you see above is actually two iFrames – the top (player) section and the bottom (album select) section.

The top section can issue AJAX commands to the backend, written in Python using the surprisingly well-documented Python-MPD2 library.

This is all pretty straightforward, but for two things: keeping the current track data fresh and accessing album covers.

Keeping track data current doesn’t have an elegant solution – I have the JavaScript query the backend every 4 seconds and check for differences in the track name or album artwork file. It’s not fancy, but given I can’t access MPD directly via JS, keeping it constantly up-to-date is beyond my meagre ability.

MPD supposedly allows for access to binary data containing album covers, but the rules for this are opaque and inconsistent – I have a number of albums with corresponding images in the right directories, but no artwork shows up.

MALP works around this by (semi-successfully) pulling data from MusicBrainz where possible. So I did the same – an artwork backend informs the panel of the best image to use based on existing files and downloads the appropriate album cover where artwork doesn’t already exist.

This mostly works, but is hardly robust.

Album Selection

This is mostly smoke and mirrors – I lack the patience or inclination to build a system for gathering data on all the artists or albums in our collection. Instead, I figured we only have a handful of albums at any time we’re really listening to. I’ve created stored playlists for those using MALP and another backend script runs a cron job once a day to add any new playlists to the frontend.

What was most “fun” about this was of course passing the playlist names back and forth – they’re replete with lovely juicy characters such as apostrophes and ampersands which break URLs something fierce.

As a result, the JavaScript on the front end base64 encodes the playlist names before URI component encoding the base64 string and the Python backend undoes all that to load the appropriate list.

At this point I’m remembering why I never stuck with frontend development back when I worked as a dev.

A note about JQuery:

The music player is the only part of the project to use JQuery. I resolved not to use any frameworks where possible – mostly out of bloody-mindedness, but also because it wasn’t really essential for any of the panel components – but the exception was marquee.

Yes, that marquee.

You see, for long track names, they won’t fit in the 300 odd pixels set aside for the player. So for those tracks, I wanted the name to scroll when the track was playing.

To get a reliable marquee effect in this day and age, you apparently have to use a JQuery plugin and about 20 lines of CSS.

Calendar

It’s a calendar.

One does not simply make a calendar in software. Time is evil when it comes to software development – I’m torn as to whether JavaScript or the entire concept of time itself is worse.

Okay, it’s JavaScript – I have to actually use JS more often.

Rather than reinvent the extremely complex and fraught wheel of time, we can just use Google Calendar’s “embed your calendar as an iFrame” feature. Perfect!

It looks like this:

The “embed as iFrame” feature predates the popularity of dark modes in computing.

Hmm. That doesn’t look like the rest of my panel.

A brief word about the panel’s colour scheme:

I’m not good at colour schemes. More than that, I struggle to care about them much. It’s not that I’m colour blind, I’m just largely colour agnostic. I mean, I’m not going around using hotdog stand theme or something, but beyond that, well, ¯\_(ツ)_/¯

Here’s the thing though – you can’t just restyle the contents of an iFrame when it comes from someone else’s server – there are very important security reasons for that. Given this was a project purely for use in my kitchen, on a device inaccessible to the outside world, I did look for workarounds to this security policy.

As a result, I found a lot of people discussing some super bad ideas for circumventing XSS protections, and there’s probably a fortune to be made in bug bounties if I looked up their Linkedin pages and did some half-hearted prodding on the web apps run by their companies.

So disabling XSS security is a non-starter.

Next best option: User Scripts!

You might know these as GreaseMonkey on Firefox or TamperMonkey on Chrome. They are excellent for bending the world of the web to your own twisted vision.

A basic userscript later and I’m forcing the browser to restyle the Google Calendar in a dark theme. It is absolutely, totally and in all other ways perfect, and I’ll not be moved on the issue.

Don’t click the “agenda” or “week” buttons.

What’s the weather like? Nice.

You see that moon icon? I drew that myself. Graphic design is my passion.

(You’ll note the “sunny” icon in the screenshot at the top doesn’t match this one – Chrome aggressively caches images and it’s challenging to have it download a changed version)

This was a hot mess. So to speak.

There exist APIs for weather which are free. None of the ones I found were worth a damn for local weather – often off by 2-6 degrees (43 degree day? It happily reports that it’s 37!)

There exist APIs which are very expensive. This is not suitable for my flimsy summer panel.

Then… there’s *coughs* scraping weather websites.

I won’t go into detail, but to ensure that the scraping only happens a handful of times a day, I wrote a Python script with Beautiful soup to snag the precis for today and tomorrow along with the temperatures and dump them into text files on the webserver. Then, the weather app reads that data in every 15 minutes or so.

Weather icons are usually pretty awful. I like these ones.

UV was… trickier.

When my wife asked for UV info, I said, “Easy. It’s summer, therefore it’s always extreme. Pack sunscreen and wear a space suit.”

But no, the spousal request was for moment-to-moment UV rating values (or thereabouts).

The official UV rating site for Australia was last updated when Kings of Leon were a big deal on the radio.

I guess at least Kings of Leon released new material since then?

Some digging unearthed a JSON file shuttling from a server to a script on the page and then into some terrible graphing libraries. As well as many, many commented lines of code that were clearly not meant to make it into production.

I pointed some more Python at the JSON file and… oh great, it’s a giant array of UV data for every minute of the day. I’m going to have to iterate through the whole thing and… wait, there’s a property at the bottom of the file called “current_uv_rating”, perfect!

Nope. It’s always set to zero.

Iterating it is. Another cron job runs the Python every half hour.

Have I got news for you?

Getting ahold of news headlines was surprisingly challenging too – there isn’t an API for our nationally funded provider (er, that is, nationally funded provider not directly owned by our home grown Dark Lord).

They do offer RSS feeds (am I hearing decades old music again?) which they seem to have been very enthusiastic about around the time that everyone was done using RSS feeds. Most of the info on ABC’s site regarding RSS feeds is from circa 2011, and I’m cautiously using the word “most” as that implies there’s a lot more information than there actually is. Which is close to none.

There does exist a “just in” feed – an XML file with headlines, links and relevant images for the most recent stories, regardless of topic or popularity. I don’t know what dark science or eldritch divination led me to find it, because you sure as heck can’t track it down using either Google or the ABC’s own search functionality.

At any rate, another Python script + cron job (TM) later, and I’m slurping down headlines to display every 25 seconds on the panel.

Here’s The Thing, though: when you visit an article by smearing your finger on the screen, it opens in a new tab. Which would be dandy, except the only way to get back to the panel again is to:

  1. Press the Windows symbol on the Surface device running the panel (a feature sadly missing from more modern Surface Pros) to access the taskbar, given the panel’s browser is in full screen mode.
  2. Access the on screen keyboard from the taskbar (no, not the pretty one. The accessibility one with all the functionality).
  3. Press fn-F11 to drop the browser out of full screen mode.
  4. Close the tab. Oops, you missed the little “X” with your giant finger. Try again. No, that’s a new tab. There we go.
  5. Access the keyboard again. Press fn-F11 to go back to the full screen mode.

That dog won’t hunt, Monsignor.

War were declared.

I need a big, meaty, easily touchable “close” button on articles when they open.

iFrames are once again unsuitable for this job – I could just display the article in an iFrame with a button outside it – but ABC have (entirely reasonably) prohibited using their stuff in an iFrame, specifically to prevent nefarious purveyors of stolen bits from claiming their work as their own (not that I’m a purveyor of stolen bits. I don’t purvey them, thank you very much.).

Once again we turn to… Userscripts. Hooray.

The CMS used by our National Broadcaster is, like all CMSes, prone to creating multiple obscure classes in its HTML.

Therefore finding the right element to inject a button into was tricky.

It’s not robust by any means, but it currently finds the second link on the page (the ABC logo) and sticks a close button right next door. It then tracks down the “FixedHeader” data component in the page and injects another button – this is the overlay banner that sticks around at the top once you scroll down.

Et voila – one click return to the panel.

The #bestoftimes, the #worstoftimes

Fig 1: A somewhat silly clock

This is a Javascript clock. It is pretty silly. Maybe one day I’ll replace it with a graph of utilities usage or something. It’s a nice clock though.

Fig 2: A less silly, but far more boring, clock

This is another clock. This one is more sensible and useful. I don’t know what it might do on Wednesdays in September since I didn’t use a fixed width font and it’s pretty close to the full width of its container.

I’ll be shocked if the panel is still fully working in September though.

How does this dang thing work?

It runs on an old Surface Pro, which is comically overpowered for the job it needs to do. That said, it is quite old and water damaged – part of the reason for the dark theme (apart from the fact that everyone’s doing it, it’s cool man, what are you, some kind of square?) is because water ingress damaged the screen some time ago, leaving weird blotches that are only visible when displaying bright images.

The panel itself is just some HTML with iFrames and a little JS to reload the calendar and weather. It’s actually not even hosted on the Surface, but instead sits on our media centre PC/NAS.

The media centre runs the necessary backend cron jobs and hosts the music, which is played on the third, far more interesting and terrifying PC, the MPD server.

To sum up – this runs off three physical machines, 3 cron jobs, a locally hosted site, an MPD server, two userscripts which are dependant on their target sites “not changing too much”, various Python scripts for scraping sites (which also need to not change much, please and thank you) and a Python script parsing and processing an RSS feed that its owner no longer seems to care for.

The panel’s days are numbered – everything on the web is transient, even local pages which are only used in a single household.

This is our reality – endless sandcastles

This isn’t necessarily a bad thing – a bit over a year ago, we had a Google Nest we’d received for “free”. It was fine.

It played music – but not necessarily the music we wanted or in the order we wanted. And it had ads.

It could tell you the weather, but you had to ask.

It could tell you your schedule, sort of.

It could add to our shopping list – provided you used the shopping list page Google created, which has no API.

When it stopped working, I searched for a solution and found nothing useful. It was apparently a known flaw. Had we paid for it, we’d probably have some recourse for replacing it.

I didn’t feel the need to buy another.

At least when this panel breaks, I’ll be able to find out why and have a chance at fixing it.

Bon Voyage, little kitchen bench automaton!

ISPConfig & Let’s Encrypt

The guys at ISPConfig do good work – although it can sometimes get in the way of my own server configuration, the vast majority of the time it saves a huge amount of effort on what would otherwise be mundane and routine tasks.

With FOSS solutions like this though, “must-have” features don’t always come quickly – CPanel implemented Let’s Encrypt functionality very early on, but ISPConfig users had to wait 6 months (which is pretty fast considering the circumstances).

I needed a solution immediately though, which is why I rolled this (reasonably awful) automation to integrate.

The solution has worked well enough for the past 12 months or so (although implementation on slave servers wasn’t easy or pretty), but I figured it was time to upgrade to ISPConfig 3.1 – with built-in Let’s Encrypt support.

It’s almost always best to stick with the native solution rather than a third party one (particularly when the third party one is developed by a time-poor hacker), and that rule largely applies here. Continue Reading…

Hosted Exchange – Auto-forwarding to external domains

A Very Sensible policy, enforced by default in Exchange Server is to ignore rules automatically forwarding mail to external domains.

It’s fairly easy to see why this is, in fact, Very Sensible:
Your organisation assigned email addresses to people who have agreed to be bound by its policies (right?) – allowing auto-forwarding to any address outside that domain risks you being responsible for a breach of confidence.

I’m all for having the “secure” option be the default and for discouraging or preventing users from breaking that security in the name of convenience.

But there are times when other, less sensible policies are in place that I feel the users should have recourse to implement workarounds. One such policy might be (for example), having an email quota set to (purely hypothetically), 100MB.

This is ample space for email in 1997. This is hilariously limited space in 2016. Continue Reading…

Android – Location Based Reminders

Just a quick one:

I’ve had issues with Google Now’s reminders since I first owned a Nexus 4 (a few years back) and the issue persists with my Nexus 5X. The trouble is in having reminders trigger at a particular location, rather than a particular time.

For example, if I know I need to pick something up next time I visit my in-laws, but don’t know when I’ll next be at their house, I can set a reminder and select “place” instead of time. In theory, entering their address is all that is needed to make it work.

google-now-location-b

This image was stolen wholesale from this article: http://www.makeuseof.com/tag/8-amazing-life-improving-uses-google-now-reminders/

In practice… nothing. Just… nothing. Continue Reading…

Looking for a Copy replacement: pCloud

As of this month (May, 2016), Copy is no more.
(I’m sure one day soon that link will be a dead end or be sold to someone not relevant)

Copy was a cloud storage service much like Dropbox or Google Drive – you had an amount of storage space to fill up with your junk, a web interface to access it, some local applications for your computers and mobile devices and it handled syncronisation.

there is no cloud

Sticker from Chris Watterston – click the image to visit his store thingy

Continue Reading…

Let’s Encrypt – Mysterious Authority Issues

Lately, new sites I’ve created using my ISPConfig automation and letsencrypt.sh have been received inconsistently on various browsers – the issue appears to be particularly prevalent on OSX.

Doing some digging revealed a possible incomplete chain issue to be the cause.

Sure enough, modifying my Apache conf to incorporate a direct link to the intermediate chain fixed the issue.

My LE-ISPConfig Apache conf now looks like this:

That little line at the bottom was what made the difference; chain.pem (a symlink to my primary domain’s intermediate cert chain) will be updated as and when keys and site certs are updated via cron. Continue Reading…