Python, oursql and executemany

I’ve never had cause to use executemany before, so perhaps it is unfortunate that my first encounter be in the context of oursql’s somewhat anaemic documentation.

I’ve just spent well over an hour trying to get a simple executemany statement to work – in retrospect, the solution was obvious. But the presence of an example (just one!) would have made things immediately clear.

To that end:

reportDB = oursql.connect(host = "localhost",
                          user = "LapsedPacifist",
                          passwd = "ContentsMayDiffer",
                          db = "AnythingLegalConsidered")

with reportDB as query:
  query.execute("BEGIN")
  query.execute("""
                INSERT INTO assets(clientid, assettype, model, deployed, name)
                VALUES(?, ?, ?, ?, ?)
                """, (clientID, ASSET_TYPE_UNKNOWN, model, deployed, name))

  assetID = query.lastrowid
  addresses = []

  for ip in ips:
    addresses.append([assetID, ip])

  query.executemany("""
                    INSERT INTO asset_ips(assetid, ipv4address)
                    VALUES (?, ?)
                    """, (addresses))

That’s it – just as oursql’s parameterisation is looking for a list of parameters, executemany is looking for a list of parameter lists – not just a list of strings.

Hence the for loop creating a list with the two parameters and then appending it to the executemany parameter list.

I’m sure there’s a better, more pythonic way of doing that – please let me know if I’m missing something obvious.

And behold, the grand total of what oursql’s documentation has to say about executemany:

Additionally, executemany() is lazy; if passed a generator or any other iterator which does produces values lazily, values will only be taken from the iterator immediately before they are sent to the database.

executemany(query, parambatch)
Execute the same query with different sets of parameters. This is more efficient than calling execute many times with the same query.

I don’t want it to sound like I’m down on oursql – I feel it’s the superior library for MySQL in Python, and it’s still quite young – but this particular gap in the documentation had me pulling out my hair.

I’ve read some complaints regarding scalability of executemany in oursql – fortunately my code won’t ever have to handle huge numbers of queries at one time.

Promiscuous Plink

Sometimes it’s necessary to do things the wrong way.

Plinkmiscuous is the world’s tiniest, laziest modification to the very excellent Plink, part of the PuTTY suite of software.

Plink is a command line only connection tool – it has much of the core functionality of PuTTY without all the fruit of a window and GUI. It’s often necessary to connect  through to *nix servers from Windows programmatically, and Plink is generally the tool for the job.

Unless you have a scenario where you can’t know the SSH fingerprint of the server to which you are connecting, or can’t store it (PuTTY puts things in the registry – I’m assuming a hangover from the bad old days of XP) for the account you need to run as.

In these circumstances, the otherwise fully automated Plink demands user interaction to confirm the fingerprint as OK.

Obviously, this is by design – the whole point of SSH is that you can be sure the server you’re connecting to is indeed the one you want before you start sending over authentication details.

But sometimes, security be damned, you need to do something dirty. Please don’t ever do it in a serious or production system.

To that end – the traditional way to “break” Plink and make it work regardless of a stored fingerprint is to pipe or otherwise send a “keypress” for confirmation.

That method sucks.

Plink is open source, so that means we can modify it when it fails to do what we want.

So that’s what I’ve done – it was a change of exactly 1 byte (and some Visual Studio fru fru).

You can find Plinkmiscuous on the software page. It’s based on the current version of Plink at the time of writing. I can update it in the future, but make no promises (and the code change is so simple, you should do it yourself).

And it’s probably worth rethinking whatever it is you’re planning to do with Plinkmiscuous so you don’t have to use it. :-P

Your Security Policy Sucks

I’m talking to you, enterprisey organisation.

Your ITS people just got back from a seminar and they’re all fired up to enforce password changes every 3 months and allow your users to “take control of their accounts” by using “security questions” to reset passwords.

Well that’s dumb. Don’t do either of those things.

People are bad with passwords. They don’t understand security in general. Forcing them to change their mediocre passwords every 90 days instead of never is only going to exacerbate the problem.

Let’s take a look at my university’s policy:

crap_pw_policy

  • “A maximum of 16 characters” – Don’t do this. Don’t be this dumb. Once your password is salted and hashed, it doesn’t matter if it’s 16 characters or 216 characters. This isn’t 1988 and we’re not storing plain text passwords in a non 3NF database using COBOL. Stop putting low (very low) upper limits on password length.
  • “A minimum of 1 upper/lower/number/special” – great, you’ve just narrowed things down for the guy trying to brute force a password (not that that’s how people break in these days anyway). How many of your people are going to put more than the minima you just set in their 8-16 characters? A negligible few.
  • “The password can’t contain… spaces” – Don’t. Just… don’t. Do. This. Combined with the 16 character maximum above, this completely rules out using a passphrase (longer, easier to remember). There’s no backend reason to disallow spaces, which means you either have this rule because you heard it was the right thing to do (it’s not) or because you have a badly designed front end which breaks on spaces.
  • Challenge questions

 

“Challenge questions” or “security questions” or “secret questions” or “giving out personal details to everybody for no good reason” were a kludge invented for a bygone era and are no longer useful or necessary.

In fact, they’ve been badly implemented since their inception, and were worse than useless even 7 years ago.

You might have seen in the news lately that a number of celebrities have had their iCloud accounts broken into and private photos stolen and distributed.

Why did this happen?

Well yes, because some people are complete assholes. But how it happened, dollars to doughnuts, is that many of these celebrities used questions to which the answers are not difficult to divine. It’s not their fault – they’re not the experts. You, enterprisey organisation, are. And if your security policy is only good against people who don’t actually know you, then it sucks.

See also, this blog post from iiNet.

Your password should be a minimum of 9 characters

Why 9? Is this a “You don’t have to be able to outrun the bear, just the slowest person in your camp” situation?

No white space

One character is just as good as literally any other. The only vaguely plausible argument I’ve seen to justify this is that passphrases will often have lower entropy than passwords – since people write predictable sentences as much as predictable words and sentences usually consist of fewer components (words vs characters).
Except, people who use passphrases don’t make them up. They have them randomly generated.
They’re also rare, so not worth trying to tackle.

Why is it silly to ask your users to choose meaningless strings of characters, especially in light of the fact they’ll have to change the damn thing in 3 months anyway?

Because your users can’t remember them. Many of your users don’t log in every day or even every month – the next time they log in might be to change the damn thing because 90 days are up.

So they tack a 1 to the end of their last password. Or if you forbid minor modifications (which you shouldn’t even be able to check, by the way!), then they write them down. Or both. Which defeats the purpose of your security policy, or in many cases is worse than just letting them keep the same damn thing for 8 years.

What’s the solution?

Use proper two factor authentication (no, security questions are not 2FA – at best, they’re half factor authentication).

If you’re a company, issue dongles to your users. You can even re-sync them every 90 days if you like.

If you’re not, issue a key generation app for smart phones (or any phone that runs Java) or use an existing one.

Failing (or in addition to) that, you can SMS codes to your users’ phones. Even banks manage that much, and they’re basically the gold standard in doing this stuff badly.

You can also issue security certificates for your users to combine with their passwords. Issue a new one every 90 days or monthly or whatever makes you happy – it’s nothing new for them to remember, so they won’t care.

And fire the guy who does your ITS seminars.

Logitech Restarter

This will be of interest only to those who: have a Logitech keyboard with a fancy display, put their PC to sleep from time to time.

The Logitech Restarter is a little program designed to do nothing more than kill and restart Logitech’s Gaming Software.

Hardware manufacturers are notoriously bad at making stable, usable software (barring, one hopes, drivers). LGS isn’t as awful as some offerings in the past – but one area in which it does fall down is maintaining its connection to the keyboard’s display after waking from sleep.

This little application seeks to work around that problem by automating the user’s only recourse: killing the application and starting it again.

Code, binaries and more info at my software page.