In Praise of Higher Order Functions

In February, Mary Sheeran gave a talk at LambdaDays called “In praise of Higher Order Functions and of some friends and heroes”. I was not able to attend LambdaDays, but I watched the video of the talk and I really enjoyed it.

screen shot of video playing

Sheeran is urging us to read these historic papers, but I was disappointed to be unable to find many of them on the internet. It wasn’t until her 41st slide that she referred to something that we can see unimpeded! I guess it’s not really possible for an amateur to read all of these papers. One really must belong to a university or some other institution that subscribes to these journals and has (or can borrow) these books. What a shame.

Sheeran ended her talk with an admonishment from Christopher Strachey to bring together theory and practice.

It has long been my personal view that the separation of practical and theoretical work is artificial and injurious. Much of the practical work done in computing, both in software and in hardware design, is unsound and clumsy because the people who do it have not any clear understanding of the fundamental design principles of their work. Most of the abstract mathematical and theoretical work is sterile because it has no point of contact with real computing.

Strachey said that over fifty years ago in another paper I can’t find on the internet (“Towards a formal semantics”, 1966), but it doesn’t seem like we’ve listened. I’m a working programmer and I can’t easily read this research.

Anyway, I did track down some of the papers, so I thought I’d post the links I found. Some of the links are PDFs.

John McCarthy

  • LISP Programmer’s Manual
    • Modification number 7 – March 3, 1959
    • Modification number 12 – March 20, 1959
    • I found one version (not sure which one) behind a paywall
  • LISP 1.5 Programmer’s Manual, second edition, fifteenth printing, 1985

Christopher Strachey

  • Strachey began experimenting with higher-order functions in 1961
  • Strachey developed R1 (foldr) and R2 (foldl)
  • David W. Barron reproduced the 1963 workshop from audio recordings
  • Barron and Strachey, “Programming”, 1966 (chapter three of Advances in Programming and Non-Numerical Computation, which you can buy from Elsevier)
  • Oliver Danvy and Michael Spivey, “On Barron and Strachey’s Cartesian Product Function”, 2007 (buy from the ACM).
  • Bitwise Operations, 1961 (buy from the ACM).

Kenneth Iverson

Graham Hutton

Guy E. Blelloch

Duane Merrill and Andrew Grimshaw

Horacio Gonzalez-Velez and Mario Leyton

  • A survey of algorithmic skeleton frameworks: high-level structured parallel programming enablers, 2010 (behind a paywall)
Advertisements
In Praise of Higher Order Functions

CPOSC 2017

Yesterday, I drove up to Lancaster for the Central Pennsylvania Open Source Conference. It was terrific! This is my third time doing this and it’s become one of my favorite conferences. There are three tracks of talks most of the day, so you can almost always find something interesting to learn about at every session. Often, you have to decide between two or more talks that you’d really like to see.

pic of empty stage

Don Schenck had lots of practical advice for beginners in a talk called “OMG How Do I Even Start An Open Source Project?”

Matt Rogish talked about “Kubernetes 101” and how to pronounce kubectl (“coob see tee el” or “coo bee cuttle”).

In “Kubernetes: A Deep Dive,” Mike Newswanger pronounced kubectl “cube control.”

Rogish and Newswanger both used Docker for all of their examples. They both mentioned that you could run Kubernetes without Docker, but said no more about it. I was kind of hoping to hear about using Kubernetes to orchestrate other things.

Jason O’Donnell talked about “Vault: A Tool for Managing Secrets” and had a nice demo.

James Tramel did an “Introduction to Blockchain.” This was nearly impossible to do in thirty minutes. He did a fine job, but there were clearly more questions in the room when we had to stop.

Sarah Mogin gave a really great talk called “Building for Performance and Scale with React/Redux.” I don’t usually have much patience for JavaScript, but she had ton of practical advice with no sugar coating.

“Man, JavaScript is annoying.” – @sarahmogin

Much of her talk had an A versus B structure for a variety of A’s and B’s (traditional versus headless, Component versus PureComponent, &c.) which was very effective.

pic of Sarah Mogin

I also really enjoyed Steve High‘s talk “Ditching REST for gRPC.” His demo was an arithmetic service “because who doesn’t like math.” He live-coded subtract into the service during the talk. He didn’t hear me answer “minuend” when he asked, so his variables were called firstnumber and subtrahend. Naming things is hard.

pic of Steve High

There were also lightning talks, 3D printing by PrintedSolid.com, and lots of great local food.

The folks from MajorMega had a prototype of Hyperland set up that we could try out. In addition to VR headset and controllers, it had fans for wind and a floor that tipped. This guy thinks he is riding on the back of a moving truck and shooting at other cars and things chasing him.

pic of Hyperland prototype

There were also giveaways, including a 3D Printer from LulzBot. I didn’t win anything, but that’s okay. It was fantastic day!

pic of organizers

CPOSC 2017

DevFestMD 2017

I ditched work today and drove into Baltimore for DevFestMD instead…it was a blast! It took place at Betamore at City Garage, which is a really cool space in Port Covington, just across the Vietnam Veterans Memorial Bridge. There were lots of interesting talks, plus code labs (Go, Kotlin, and more!).

pic of Sarah Jennings

Sarah Jennings (@ThatOtherSarahJ) warned us that the Blockchain is coming for us!

pic of block chain slide

Danny Blue (@dee_bloo) talked about packaging Angular modules.

Mike Talbott (@MikeTalbott) told us about recent advances in augmented reality.

pic of Mike Talbott

His company, BaltiVirtual, is one of the occupants of City Garage and they generously offered tours of their spaces and demos of cool augmented and virtual reality things throughout the day.

Sal Hernandez (@clickclickonsal) expo-unded (Ha! See what I did there?) on React Native.

Chida Sadayappan (@schida) broke down machine learning opportunities.

Gavin Cannizzaro ranted about visibility in program design.

Shannon Foster (@SheCanTech) explained civic tech.

pic of Shannon Foster

At the end, they had a drawing for some cool gadgets. I didn’t win anything, but that’s okay. It was a fun day!

DevFestMD 2017

Autopsy in Xubuntu 17.10

Well, I’m happy to report that I upgraded to Xubuntu 17.10 and Autopsy still works. That’s not surprising (it’s just Java, after all), but it still might be worth mentioning.

And I managed to process some data. At first, I made a dd image of a thumb drive because Autopsy couldn’t access it directly. That worked. It was my Strawberry Perl stick and Autopsy found over 6000 email addresses (presumably all of the module authors and other contributors to Strawberry Perl), but nothing else of interest.

Next, I grabbed a random thumb drive from my backpack and ran Autopsy as root. That way it could process the thumb drive directly, without making an image first. That drive contained a Linux driver, a zip file of all the slides from last year’s Enfuse conference, a tarball containing an older version of dd_rescue, an /etc/hosts file from my home network, a directory of photos from a trip to Ottawa, a PDF of a boarding pass for a plane trip, and an empty directory.

$ ls /media/tim/oylenshpeegul
agere_sta_fw.bin
_ALL SPEAKER PRESENTATIONS IN PDF.zip
dd_rescue-1.23.tar.gz
hosts
Ottawa 2013
Southwest_Airlines_-_Print_Boarding_Passes_and_Security_Document.pdf
System Volume Information

Autopsy found 110 email addresses from the zip file and one from the tarball. It found the EXIF data from the photos. The timeline showed a history for dd_rescue going back to the year 2000. It was easy to generate an HTML report

screenshot of Autopsy HTML report

and there were a half dozen other report formats as well. All in all, a pretty good experience! Now, how to write a plugin…

Autopsy in Xubuntu 17.10

Autopsy 4.5 works in Linux!

Ever since Autopsy 3.0 was released and right up through Autopsy 4.4.1, it only worked in Windows. I could get it to compile without error on my Linux machine, but running it would only display the splash screen of the little doggie and then immediately crash.

Screen shot of Autopsy 4.4.1 splash screen in Linux

But yesterday at OSDFCon, my friend Ben said he got the newly released Autopsy 4.5 to compile and run on his Ubuntu 16.04 machine! The weird part was that he said he didn’t do anything different, it just worked.

Today, I tried compiling it on my Xubuntu 17.04 machine and it did indeed work! It wasn’t completely straightforward, so here’s what I did:

First, get Sleuthkit 4.5…

$ cd ~/sleuthkit
$ tar xf ~/Downloads/sleuthkit-sleuthkit-4.5.0.tar.gz 
$ cd sleuthkit-sleuthkit-4.5.0

…and build it. It didn’t have a configure file included, so we’ll need need to run autoconf ourselves.

$ autoreconf --install
$ autoconf
$ ./configure
$ make
$ make check
$ sudo make install

Now we have to build the Java bindings for it.

$ cd bindings/java
$ ant dist-PostgreSQL

But Autopsy will be looking for Tsk_DataModel_PostgreSQL.jar rather than Tsk_DataModel.jar, so we’ll fake one of those.

$ cd dist
$ ln -s Tsk_DataModel.jar Tsk_DataModel_PostgreSQL.jar

Okay, now we’re ready to get and build Autopsy 4.5.

$ cd ~/sleuthkit
$ tar xf ../Downloads/autopsy-autopsy-4.5.0.tar.gz
$ cd autopsy-autopsy-4.5.0
$ export TSK_HOME=$HOME/sleuthkit/sleuthkit-sleuthkit-4.5.0
$ ant

Now when we run it…

$ ant run
...
BUILD SUCCESSFUL
Total time: 19 minutes 25 seconds

…we eventually get an Autopsy window!

Screen shot of Autopsy 4.5 in Linux

We can try to create a new case…

Screen shot of Autopsy 4.5 create case

…and process it.

Screen shot of Autopsy 4.5 processing

Hrm. The default settings apparently include things that only work in Windows.

So it’s still pretty aggressively single-platform, but at least it runs without crashing! To be fair, they describe it as “a Windows-based desktop digital forensics tool” right on the tin. But it’s written in Java, so there’s no good reason that it shouldn’t just work everyhere.

Assuming I can get it to actually process some data for me, next I want to try writing a plug-in for it in Clojure. I bounced that idea off of Brian Carrier and Richard Cordovano at OSDFCon yesterday. Both seemed skeptical.

Autopsy 4.5 works in Linux!

Clojure/conj 2017 in Baltimore

pic of 10th anniversary cake

I just got back from Clojure/conj and wow, was it great! I don’t use Clojure professionally, but the conference was right here in Baltimore this year, so I couldn’t pass it up.

Clojure 1.0 came out in 2009, but they’re calling this the 10th anniversary because it had actually gone public a couple of years before that. In fact, one of the highlights of the conj was this morning’s talk by fogus and Chouser, where they reminisced about the whole ten years. This included details from 2007 and 2008 which were new to many of us.

At the party last night, I was able to meet Rich Hickey. I shook his hand and said, “thank you.” I’m not sure if I said anything coherent after that, so he probably thinks I’m an idiot, but I’m grateful for the opportunity nonetheless. He has done so much for programmers everywhere! He didn’t just create Clojure, he challenged the status quo. Object-oriented programming dominated our collective thinking for decades, almost to the exclusion of everything else. Now it’s common to hear people speak about alternatives.

I met lots of other Clojurists from near (Baltimore, Brooklyn, &c.) and far (Toronto, Tokyo, &c.) and every one was super nice. I’m not sure if programming Clojure makes you a nicer person or if nice people are just attracted to Clojure, but either way it made for a very pleasant conference.

Clojure/conj 2017 in Baltimore

Command-not-found in Debian

pic of fortune cookie

I wrote this up years ago, but with the demise of CrunchBang those bits are rotting away. The URL is still good, but my post is mostly gone. I just installed Debian Stretch on my Mom’s old MacBook and the issue is still there, so here it is again.

If you invoke a command that doesn’t exist, most shells will simply tell you so.

$ play
play: command not found

Ubuntu has a utility called command-not-found. If you invoke a command that is not installed, but is available, it will tell you how to install it.

$ play
The program 'play' is currently not installed. You can install it by typing:
sudo apt install play

Nice!

This command is now available in other Debian systems too, including the latest stable: stretch.

sudo apt install command-not-found
sudo apt update
sudo update-command-not-found

But it doesn’t quite work right, as is.

$ play
The program 'play' is currently not installed. To run 'play'
please ask your administrator to install package 'play'

But I am the administrator! Why doesn’t it think so? It seems that Ubuntu uses different groups than Debian. In particular, command-not-found is looking for membership in group “admin”, but Debian doesn’t use that. We want it to check for membership in group “sudo” instead. We can fix that!

sudoedit /usr/share/command-not-found/CommandNotFound/CommandNotFound.py
# change "admin" to "sudo" on line 93

Most programs transition from Debian to Ubuntu. This one transitioned the other way. I guess the trip isn’t as smooth, since this was years ago and it’s still not fixed (indeed, it’s still line 93). I think this affects at least Debian wheezy, jessie, and stretch. It looks like it’s fixed on github and in Ubuntu (checks for either “admin” or “sudo”), but has not been backported to Debian yet.

Command-not-found in Debian