Blackhat 2017: Using Machine Learning to help your malware evade Machine Learning based malware detection

The present is now so futuristic it’s hilarious. Hopefully someone uploads this talk somewhere. If you’re lucky enough to be at Blackhat then check out the talk here

Mini book review: Surely You’re Joking Mr. Feynman

Who?

If you don’t know Richard Feynman, you’re in for a treat. He has done a lot of unbelievable shit: He won the Nobel prize in Physics, he worked with Oppenheimer on the Manhattan Project, he lectured alongside Einstein at Princeton and he played a key role in determining the failings that led to the Challenger disaster.

What’s it about

Well, it’s not about Physics.

It’s a selection of anecdotes from Feynman himself, jumping back and forth in time. It doesn’t cover his time in the Challenger investigation but a lot of key events in his life are talked about in detail. The book has a bit of a disjointed format. I wasn’t sure if I’d like the lack of structure at first but I found it refreshing as I made my way through. One chapter might be about the friction between him and his military bosses and the next about his bongo playing adventures in Rio.

The one thing that is consistent is his simple presentation and hilarious tone. Despite understanding and developing incredibly difficult concepts, he had an amazing ability to explain things in a way a child could understand. In fact, he’d point out that not being able to do that means you really don’t understand the thing you’re talking about at all! I remember when I was younger my brother (a big Feynman fan) always asking me to explain what I had learned in school that week. If I couldn’t do it without tripping over, he knew I hadn’t really learned it.

My only real criticism is that some of his chat comes across as dated. His attitude towards women at times is probably a bit controversial at best. He does acknowledge it though, and for the time I think he was probably pretty progressive. Reading it in 2017 though can induce a bit of cringe.

Why should I read it?

If you’re on this blog, you probably develop or are interested in developing software. Anyone who follows the Feynman method is going to improve their dev skills.

He also shows how it’s a great thing to not be afraid of looking like an idiot. Just because everyone else in the room seems to understand something, doesn’t mean they do and also doesn’t mean you should. Likewise, just because everyone else in the room does something a certain way, doesn’t mean it’s the right way to do it.

He didn’t respect authority just for the sake of it and gives numerous examples of how an ego has gotten in the way of many solutions to problems. He talks about his experiences with imposter syndrome and having confidence in your position in life.

Pretty much all of the book (except the bits about women and bongos!) can be applied to software development, especially for the guys and girls writing the code. You’ll also laugh out loud at bits.

My rating

9/10

Get it here

Vimwiki wiki wild wild west: Tables

Vimwiki

Vimwiki is awesome and it has already done the rounds a few times. If you haven’t heard of it then check it out on the Vimwiki github repository. It’s great for note-taking, to-do lists, meeting minutes, etc. but I came across a use for it this week that I’ve already found myself using a few times since: Tables.

Vimwiki Tables

I’ve used the Tabular vim plugin on and off but I find it a bit tedious. It could well be that my workflow with it isn’t great. Vimwiki’s tables in comparison are super intuitive, to me at least. They have auto-formatting just when needed but also have the power of vim actions.

Demo

The Github repo has all the info you need to create and edit some tables but, as always, I find gifs speak louder than words:
Below, I’ve created a table with 3 columns and 2 rows with the command :VimwikiTable 3 2

Then I can start filling the table with some info about my favourite people ever:

See how it formats as I go? It even adds a new row when I tab on the last cell.

Lastly, I’ve added some incorrect info for Voltaire’s birth year. I can correct that pretty easily by using the \ text object which represents a cell. So, change in cell ci\ deletes the inner contents of the cell and puts me into insert mode, ready to write the correct year:

Too easy.

Remember, check out the Vimwiki github repository docs for more commands.

Belfast and the cybersecurity buzz

Awesome to see cybersecurity growing in my home town. There’s such a wealth of skills here and I’m excited to be in the midst of it.

If you’re not from Belfast but interested then check out this great write up:

http://www.huffingtonpost.com/entry/cyber-security-in-belfast-an-industrial-reinvention_us_59079402e4b03b105b44bb16

With the World Cyber Security Technology Research Summit and the OWASP AppSec Europe 2017 conferences both being held here this week, the buzz is real.

Google gets a lot of flack these days but they still have some great innovations. They’ve just combined two of my favourite things: Machine Learning & Drawing.

Check out Autodraw

Quick linux tip

Linux never ceases to amaze me. I’ve been using it daily for years and somehow I haven’t come across this gem until now.

If you surround your terminal command with parentheses () then the command will execute and then return you back to your original directory.

Below I’ve got a really simple example. I’m in the folder “brackets” and I’ve created a folder “test”. I then use my parentheses around the command which moves into the test directory, creates a new file and lists the directory contents (showing the new file). Whenever the command finishes, I’m back in the folder I started in.

Linux terminal parentheses

Surrounding Linux terminal commands with brackets

Non-programming activities that make for a better developer

I should probably start with a disclaimer. This post contains a list of non-CS activities that have had a positive impact on my career as a developer. Some of it will work for everybody and some won’t. A lot of it is also useful for people in other fields too but I guess they’re unlikely to be reading this.

In no particular order…

Read

Reading is a great pastime and, like most of these points, should be done for its own merit. Being well read provides new insights into the world. Read Feynman on physics and you’ll discover how to think through a problem. Read about the Civil Rights Movement and be inspired by people that could stand by the courage of their convictions, despite overwhelming adversity. Obviously I’m not comparing the struggles of oppressed people to software developers (!) but my point is that the more you learn in life, the more ammo you have when faced with any challenge.

What to read though?

This is going to vary wildly depending on what your goals are but I think there are two golden rules:

  • Read as much as you can. If I could recommend one non-technical book for programmers, I’d say Carnegie’s “How To Win Friends And Influence People”. Despite having a slightly sociopathic sounding title, it’s a classic that will help improve the lives of everyone around you in a genuine and meaningful way.
  • Include a healthy dose of things you don’t agree with. Challenging your existing viewpoints is a great way to grow as a person and teaches humility.

Exercise (and eat well!)

Life is easier when you’re stronger and fitter. There’s no reason not to strive for more of both. Exercise releases dopamine and all the feel good chemicals to elevate mood. I find that I’m more focused when I’m training well and eating healthily. The sugar hit from junk food might get you through that hackathon but eat it day in day out and it’ll just leave you feeling moody and lethargic.
Pointing out that “sitting is the new smoking” is cliché these days but that doesn’t mean it’s wrong. I’ll mention some ways to counteract the effects of sitting in the next section but a comprehensive resistance training routine will go a long way also. Performing “ass-to-grass” squats and other compound exercises with good form by themselves will promote better mobility, helping the 9-5 body move beyond its restricted range of motion. The great Pavel Tsatsouline recommends 30 seconds of kettlebell swings for every hour of sitting – please learn how to do them from a good coach first however! That brings me onto my next point

Gainz

  • Find a well recommended personal trainer. I can’t emphasise this enough.
  • There are some great apps to help. Check out the Stronglifts app for iPhone and Android but make sure you get someone to check your form is solid before upping the weights.
  • If resistance training isn’t your thing, I think you’re missing out but it’s hard to go wrong with just moving more. If you want to get better at running then check out the Couch to 5k app or see if you have a local Parkrun nearby.
  • There are some great fitness related subreddits: /r/bodyweightfitness is one of the best, especially if you’re light on equipment and don’t fancy joining a gym. Some of the subscribers have created a great supplementary app containing the official recommended routine and made it available on the Play store.
  • I’ll say it one last time to drill the point home: please get a professional/expert to help with your form, especially at the beginning.

Stretch

Obviously related to the last suggestion. Postural issues are rampant in the software industry. Unfortunately there’s no quick fix but as long as you put in consistent effort, even just a little at a time, the benefits are pretty extraordinary. I experience next to no niggles from physical activity now and improved posture and mobility has had a definite positive impact on my mental wellbeing as well.

More than touching toes

  • Get one or two solid stretches for each problem area.
  • If you work at a desk I’d put money on you needing to improve your hips, glutes, back (flexion and definitely extension) and shoulders. Probably ankles too.
  • Strength and flexibility go hand in hand when done right but if you weight train and never stretch then you probably need even more help – ease into it!
  • Some people hate stretching. I personally do it while I’m watching a TV show and that works for me. There are plenty of avenues to explore. Maybe yoga is more your thing? Find what works for you.
  • If I had to recommend the most “bang for your buck” stretches I’d say: pigeon stretch, couch stretch, shoulder dislocates (with a resistance band and eventually a stick/bar), sitting at the bottom of a bodyweight squat and back bridges.
  • Check out Kelly Starrett at MobilityWOD.

Meditate

There’s a lot of pseudo science and spirituality attached to meditation which I’m not a fan of but ignoring all that it’s a great tool. Mindfulness has been shown to improve mood, help battle depression and to lead to an increase in concentration. Mindfulness sort of goes hand in hand with mediation although you could argue that mindfulness has a broader scope. Either way, I’ve found that the practice helps me stay in “the zone” for longer periods of time. This obviously applies to all aspects of life but I think it’s especially great for programming because of the high levels of focus required.

I’m not your Buddha, guy

  • This is another activity that has some decent apps to help out. Headspace is very popular, especially for guided meditations. I use “Meditation Timer & Log” on the Play store. It’s pretty basic but lets me set some useful options and keeps track of how much I’ve done.
  • Reddit to the rescue again. /r/meditation is very active and super patient to beginners – as you’d expect from a bunch of people who meditate a lot I guess!

Memory training

This is something I’ve picked up over the last few months. In all honesty, I mostly use it to learn stuff that isn’t work related but that doesn’t mean it isn’t suited for CS studies or work too. If you think you have a bad memory then you’re almost certainly wrong. If you’ve read about people with photographic memories then that’s mostly bullshit too. There are a lot of techniques to retaining information and some people pick them up by chance. The good news is that anyone can learn them at any stage in life. Even if you “naturally” have a good memory then chances are you’re can hone the skills you have quite considerably using these simple techniques.

30 days hath September

  • Spaced Repetition. There’s a fair bit of research to show that the best time to remind yourself of something is just before you forget it. Each time you do this you’ll remember the thing for a longer period of time. Eventually you’ll only need to remind yourself every few months/years to keep it in your long term memory. Once again, there’s a bit of software to help: check out Anki (web app and iPhone/Android apps). It will send you notifications to remind you to check information you want to remember (saved as flashcards). You can also download a ton of prepared decks from their site if you don’t fancy making your own.
  • Using your senses. I won’t try and pretend like I know anything about neuroscience but from what I gather tying information to one or more senses in your mind makes a deeper connection in your brain. Also, the more unusual the sensory experience then the easier it is for you to recall it. For example, say you need to remember that a Lambda is another name for an anonymous function but you just can’t make it stick. Try picturing a little lamb running around in a green field being chased by your dad. You can smell the grass and hear your dad scream “Come back you fecking lamb!”. The lamb gets closer and closer to you and right before it knocks you over you see that it’s wearing the Guy Fawkes mask from V for Vendetta that appears in all the Anonymous videos. Obviously that example is ridiculous. It’s meant to be! Next time you hear/see lambda that image will start to play in your head. Even the fact that you sat and conjured up some ridiculous story means you’ve given that fact more attention than you normally would – that in itself will help you remember.
  • Memory palaces (loci). You’ve probably heard of these and they’re a highly recommended tool although I personally haven’t made much use of them. They’re suited to learning things in a certain order like speeches.
  • Major system and Peg system. I’ll not go into depth here but I recommend picking up a decent book. Harry Lorayne’s Ageless Memory is great and covers these well. Essentially you can translate numbers to words using these two systems. With the words then you create vivid pictures that make learning number based facts easy. They’re surprisingly versatile too since a lot of related facts can be strung together using numbers e.g. I learnt the US presidents in no time by using the major/peg systems to learn when the presidents were in office (by year and position).
  • Combine the above! These techniques/tools are all complimentary to each other. Some are better suited to certain types of information but generally a mix and match works best unless the thing you want to learn is very simple. I mentioned that I learnt the presidents. To do that I used a combination of every point above (except for loci). I would find out the president’s position in the order and the years they were in office, then I’d make a flashcard in anki, create pegs out of the numbers and make a strong sensory image connecting the pegs to something about the president’s name.

Something creative

I choose to draw and I’m pretty bad at it but I love it and it gets my brain working in a different way. Again, whatever you choose should be something you will enjoy.

*insert creative subtitle here*

  • Draw, play, sculpt, dance, whatever. Find something, preferably not related to a computer of any kind, and get on with it! Do it every day, even if only for 5 minutes.

Diversify

Although I’d recommend doing all of these things multiple times a week, I’d also encourage mixing it up. Science has shown that new experiences are recorded differently in memory. You know the way life seems to fly by quicker as you get older? When you’re young, a lot of things are new but as you get older your experiences are usually things you’ve done before. Our brain filters out familiar memories almost like you never experienced them. By doing new things, you not only grow as a person but your life will even seem longer when you look back on it.

Sleep

Alright I’m just taking the proverbial now right? Nope. A good sleep not only feels awesome and helps with important bodily functions like hormonal balance; it also supports pretty much every other activity mentioned in this post! Exercising needs a recovery period for optimal efficiency. Same goes for stretching. A well rested brain is more able to digest and remember information. You get the gist…

You may have noticed that these things can all be done without going near an electronic device. That is by design. Developing is often a series of “Aha!” moments (in between long periods of “I’m an idiot” moments) and experience has told me that achieving that “Aha!” is usually done after taking a break from the problem.

Have fun, I’m heading outside.

From this year to next with Python’s Arrow

Date/time done (almost) right

Anyone who has dealt with timezones in their software knows how much of a nightmare it can be. Anyone who has used Python for time handling will also find out that it’s a bit of a minefield. Since we’re almost in 2017, I thought I’d (belatedly) add to the Arrow hype train. Arrow offers a more Python-like experience when writing code that deals with time or timezones. It’s simple but concise*.

The code

As always, a demo is better than a thousand words.
Hopefully the code doesn’t need much explanation. We call now() if we want a local time arrow instance. now(‘%timezone%’) if we want the same in a different timezone. humanize() is a very cool sweetener that translates the arrow object into a more human readable format:

Here’s what we’d get as output:

Search and find instance

A nice little touch is the ability to grab a time from a string. The given example from the docs shows this off better than I could:
found_time = arrow.get('June was born in May 1980', 'MMMM YYYY')

Arrow will correctly extract “May 1980” based on the pattern and ignore “June”. There’s more over at the API docs.

Unfortunately this awesome library won’t stop me from writing 2016 everywhere for the next few weeks but for everything else it does the job.

* As with all timezone libs, they made a few mistakes that will draw criticism (some dodgy naming and the overly ambitious “get()”)

Python 3.6 brings smarter text formatting

Python 3.6 is out!

New major release of Python today brings a few handy upgrades.

I particularly fancy the F string formatting. I’ve always liked how simple formatting of strings is in Python but it got even easier. I’ll let you RTFM if interested but essentially you stick an ‘f’ in front of a string literal to perform a more succinct version of .format(). Pretty cool.

Good review of AI research and where it might lead

If you’re interested in getting an overview of AI research and possible future trends check out Building Machines That Learn And Think Like People and the previous post on the same blog Artificial Intelligence And Life In 2030.

In fact even if you’re not fussed on AI the whole blog is great if you’re into general CS research. The author, Adrian, takes a new research paper every weekday morning and reviews it.