Programming Rust

cover of Programming Rust book

I’m currently reading Programming Rust by Jim Blandy and Jason Orendorff. I’m really enjoying it.

I bought it after hearing this talk by Orendorff. I loved the talk, so I bought the book. I figured if he can put together a talk that good, he can probably put together a pretty good book too. And I was right!

More recently, I listened to this podcast with Blandy. I really enjoyed this as well. So it’s no wonder I’m enjoying the book. I have no idea how they divided the writing duties, but it doesn’t matter— I like both styles.

Advertisements
Programming Rust

Autopsy 4.7 in Linux

Last year, I got Autopsy 4.5 to work in Linux, including processing some data. Now, Autopsy 4.7 is out and it claims even better Linux support. Today, I decided to try it.

First, I’ll uninstall the existing sleuthkit.

$ cd ~/sleuthkit/sleuthkit-sleuthkit-4.5.0
$ sudo make uninstall
...

Now, I’ll follow the instructions for Linux

$ sudo apt install libvhdi1 libvmdk1 libvhdi-dev libvmdk-dev libpostgresql-jdbc-java libc3p0-java
...
$ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
$ sudo dpkg -i ~/Downloads/sleuthkit-java_4.6.1-1_amd64.deb
...
$ cd ..
$ unzip -l ~/Downloads/autopsy-4.7.0.zip
$ cd autopsy-4.7.0/bin
$ ./autopsy 
$ chmod +x autopsy
$ ./autopsy 
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.netbeans.ProxyURLStreamHandlerFactory (file:/home/tim/Autopsy/autopsy-4.7.0/platform/lib/boot.jar) to field java.net.URL.handler
WARNING: Please consider reporting this to the maintainers of org.netbeans.ProxyURLStreamHandlerFactory
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Library not found in jar (libtsk_jni)
SleuthkitJNI: failed to load libtsk_jni

Hrm. It didn’t work. The graphical window says this

org.netbeans.InvalidException: StandardModule:org.sleuthkit.autopsy.core jarFile: /home/tim/Autopsy/autopsy-4.7.0/autopsy/modules/org-sleuthkit-autopsy-core.jar: java.lang.UnsatisfiedLinkError: org.sleuthkit.datamodel.SleuthkitJNI.getVersionNat()Ljava/lang/String;

So where is libtsk_jni?

$ dpkg -l | grep sleuth
ii  sleuthkit-java                          4.6.1-1                                    amd64        tools for forensics analysis on volume and filesystem data

$ dpkg --contents sleuthkit-java_4.6.1-1_amd64.deb
drwxr-xr-x root/root         0 2018-05-08 11:38 ./
drwxr-xr-x root/root         0 2018-05-08 11:38 ./usr/
drwxr-xr-x root/root         0 2018-05-08 11:38 ./usr/lib/
drwxr-xr-x root/root         0 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/
-rw-r--r-- root/root    935224 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk.so.13.4.2
-rw-r--r-- root/root     75042 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk_jni.a
-rw-r--r-- root/root   1719320 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk.a
-rw-r--r-- root/root    133480 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk_jni.so.0.0.0
-rw-r--r-- root/root      1174 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk.la
-rw-r--r-- root/root      1225 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk_jni.la
drwxr-xr-x root/root         0 2018-05-08 11:38 ./usr/share/
drwxr-xr-x root/root         0 2018-05-08 11:38 ./usr/share/java/
-rw-r--r-- root/root   5131696 2018-05-08 11:38 ./usr/share/java/sqlite-jdbc-3.8.11.jar
-rw-r--r-- root/root   1399359 2018-05-08 11:38 ./usr/share/java/sleuthkit-4.6.1.jar
drwxr-xr-x root/root         0 2018-05-08 11:38 ./usr/share/doc/
drwxr-xr-x root/root         0 2018-05-08 11:38 ./usr/share/doc/sleuthkit-java/
-rw-r--r-- root/root       512 2018-05-08 11:32 ./usr/share/doc/sleuthkit-java/copyright
-rw-r--r-- root/root       196 2018-05-08 11:33 ./usr/share/doc/sleuthkit-java/changelog.Debian.gz
lrwxrwxrwx root/root         0 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk.so.13 -> libtsk.so.13.4.2
lrwxrwxrwx root/root         0 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk_jni.so -> libtsk_jni.so.0.0.0
lrwxrwxrwx root/root         0 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk.so -> libtsk.so.13.4.2
lrwxrwxrwx root/root         0 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk_jni.so.0 -> libtsk_jni.so.0.0.0

$ ls -l /usr/lib/x86_64-linux-gnu/libtsk*
-rw-r--r-- 1 root root 1719320 May  8 11:38 /usr/lib/x86_64-linux-gnu/libtsk.a
-rw-r--r-- 1 root root   75042 May  8 11:38 /usr/lib/x86_64-linux-gnu/libtsk_jni.a
-rw-r--r-- 1 root root    1225 May  8 11:38 /usr/lib/x86_64-linux-gnu/libtsk_jni.la
lrwxrwxrwx 1 root root      19 May  8 11:38 /usr/lib/x86_64-linux-gnu/libtsk_jni.so -> libtsk_jni.so.0.0.0
lrwxrwxrwx 1 root root      19 May  8 11:38 /usr/lib/x86_64-linux-gnu/libtsk_jni.so.0 -> libtsk_jni.so.0.0.0
-rw-r--r-- 1 root root  133480 May  8 11:38 /usr/lib/x86_64-linux-gnu/libtsk_jni.so.0.0.0
-rw-r--r-- 1 root root    1174 May  8 11:38 /usr/lib/x86_64-linux-gnu/libtsk.la
lrwxrwxrwx 1 root root      16 May  8 11:38 /usr/lib/x86_64-linux-gnu/libtsk.so -> libtsk.so.13.4.2
lrwxrwxrwx 1 root root      16 May  8 11:38 /usr/lib/x86_64-linux-gnu/libtsk.so.13 -> libtsk.so.13.4.2
-rw-r--r-- 1 root root  935224 May  8 11:38 /usr/lib/x86_64-linux-gnu/libtsk.so.13.4.2

That seems right…why can’t it find it? Oh, well; forget the package…

sudo dpkg -r sleuthkit-java
sudo dpkg --purge sleuthkit-java

Let’s try building the entire sleuthkit from source

$ cd ~/sleuthkit
$ tar xf ~/Downloads/sleuthkit-4.6.1.tar.gz
$ cd sleuthkit-4.6.1

# Hey, there's a configure script already!
$ ./configure
$ make
$ make check
$ sudo make install

# All set?
$ which mmls
/usr/local/bin/mmls
$ ls bindings/java/dist/
sleuthkit-4.6.1.jar

$ cd ~/sleuthkit
$ unzip ~/Downloads/autopsy-4.7.0.zip
$ cd autopsy-4.7.0/
$ bash unix_setup.sh
photorec found
Java found in /usr/lib/jvm/java-8-openjdk-amd64
/usr/local/share/java/sleuthkit-4.6.1.jar found
Copying into the Autopsy directory
Autopsy is now configured. You can execute bin/autopsy to start it

$ bin/autopsy
...
SleuthkitJNI: loaded libtsk_jni
...

Success! Aw, but when I started a new case, it crashed!

screen shot of creating case

The pop-up window said

Sorry, the application java has stopped unexpectedly.

If you notice further problems, try restarting the computer.

and the terminal said

# A fatal error has been detected by the Java Runtime Environment:
...
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %d %P" (or dumping to /home/tim/core.21205)
#
# An error report file with more information is saved as:
# /home/tim/hs_err_pid21205.log
...
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
...

So. I don’t really know what to do about this setback. This Ubuntu machine is now at 18.04, but I doubt that’s the problem. I’m using OpenJDK, not Oracle Java…but if that’s the issue, then I don’t want to play.

Autopsy 4.7 in Linux

In Praise of Higher Order Functions

In February, Mary Sheeran gave a talk at LambdaDays called “In praise of Higher Order Functions and of some friends and heroes”. I was not able to attend LambdaDays, but I watched the video of the talk and I really enjoyed it.

screen shot of video playing

Sheeran is urging us to read these historic papers, but I was disappointed to be unable to find many of them on the internet. It wasn’t until her 41st slide that she referred to something that we can see unimpeded! I guess it’s not really possible for an amateur to read all of these papers. One really must belong to a university or some other institution that subscribes to these journals and has (or can borrow) these books. What a shame.

Sheeran ended her talk with an admonishment from Christopher Strachey to bring together theory and practice.

It has long been my personal view that the separation of practical and theoretical work is artificial and injurious. Much of the practical work done in computing, both in software and in hardware design, is unsound and clumsy because the people who do it have not any clear understanding of the fundamental design principles of their work. Most of the abstract mathematical and theoretical work is sterile because it has no point of contact with real computing.

Strachey said that over fifty years ago in another paper I can’t find on the internet (“Towards a formal semantics”, 1966), but it doesn’t seem like we’ve listened. I’m a working programmer and I can’t easily read this research.

Anyway, I did track down some of the papers, so I thought I’d post the links I found. Some of the links are PDFs.

John McCarthy

  • LISP Programmer’s Manual
    • Modification number 7 – March 3, 1959
    • Modification number 12 – March 20, 1959
    • I found one version (not sure which one) behind a paywall
  • LISP 1.5 Programmer’s Manual, second edition, fifteenth printing, 1985

Christopher Strachey

  • Strachey began experimenting with higher-order functions in 1961
  • Strachey developed R1 (foldr) and R2 (foldl)
  • David W. Barron reproduced the 1963 workshop from audio recordings
  • Barron and Strachey, “Programming”, 1966 (chapter three of Advances in Programming and Non-Numerical Computation, which you can buy from Elsevier)
  • Oliver Danvy and Michael Spivey, “On Barron and Strachey’s Cartesian Product Function”, 2007 (buy from the ACM).
  • Bitwise Operations, 1961 (buy from the ACM).

Kenneth Iverson

Graham Hutton

Guy E. Blelloch

Duane Merrill and Andrew Grimshaw

Horacio Gonzalez-Velez and Mario Leyton

  • A survey of algorithmic skeleton frameworks: high-level structured parallel programming enablers, 2010 (behind a paywall)
In Praise of Higher Order Functions

CPOSC 2017

Yesterday, I drove up to Lancaster for the Central Pennsylvania Open Source Conference. It was terrific! This is my third time doing this and it’s become one of my favorite conferences. There are three tracks of talks most of the day, so you can almost always find something interesting to learn about at every session. Often, you have to decide between two or more talks that you’d really like to see.

pic of empty stage

Don Schenck had lots of practical advice for beginners in a talk called “OMG How Do I Even Start An Open Source Project?”

Matt Rogish talked about “Kubernetes 101” and how to pronounce kubectl (“coob see tee el” or “coo bee cuttle”).

In “Kubernetes: A Deep Dive,” Mike Newswanger pronounced kubectl “cube control.”

Rogish and Newswanger both used Docker for all of their examples. They both mentioned that you could run Kubernetes without Docker, but said no more about it. I was kind of hoping to hear about using Kubernetes to orchestrate other things.

Jason O’Donnell talked about “Vault: A Tool for Managing Secrets” and had a nice demo.

James Tramel did an “Introduction to Blockchain.” This was nearly impossible to do in thirty minutes. He did a fine job, but there were clearly more questions in the room when we had to stop.

Sarah Mogin gave a really great talk called “Building for Performance and Scale with React/Redux.” I don’t usually have much patience for JavaScript, but she had ton of practical advice with no sugar coating.

“Man, JavaScript is annoying.” – @sarahmogin

Much of her talk had an A versus B structure for a variety of A’s and B’s (traditional versus headless, Component versus PureComponent, &c.) which was very effective.

pic of Sarah Mogin

I also really enjoyed Steve High‘s talk “Ditching REST for gRPC.” His demo was an arithmetic service “because who doesn’t like math.” He live-coded subtract into the service during the talk. He didn’t hear me answer “minuend” when he asked, so his variables were called firstnumber and subtrahend. Naming things is hard.

pic of Steve High

There were also lightning talks, 3D printing by PrintedSolid.com, and lots of great local food.

The folks from MajorMega had a prototype of Hyperland set up that we could try out. In addition to VR headset and controllers, it had fans for wind and a floor that tipped. This guy thinks he is riding on the back of a moving truck and shooting at other cars and things chasing him.

pic of Hyperland prototype

There were also giveaways, including a 3D Printer from LulzBot. I didn’t win anything, but that’s okay. It was fantastic day!

pic of organizers

CPOSC 2017

DevFestMD 2017

I ditched work today and drove into Baltimore for DevFestMD instead…it was a blast! It took place at Betamore at City Garage, which is a really cool space in Port Covington, just across the Vietnam Veterans Memorial Bridge. There were lots of interesting talks, plus code labs (Go, Kotlin, and more!).

pic of Sarah Jennings

Sarah Jennings (@ThatOtherSarahJ) warned us that the Blockchain is coming for us!

pic of block chain slide

Danny Blue (@dee_bloo) talked about packaging Angular modules.

Mike Talbott (@MikeTalbott) told us about recent advances in augmented reality.

pic of Mike Talbott

His company, BaltiVirtual, is one of the occupants of City Garage and they generously offered tours of their spaces and demos of cool augmented and virtual reality things throughout the day.

Sal Hernandez (@clickclickonsal) expo-unded (Ha! See what I did there?) on React Native.

Chida Sadayappan (@schida) broke down machine learning opportunities.

Gavin Cannizzaro ranted about visibility in program design.

Shannon Foster (@SheCanTech) explained civic tech.

pic of Shannon Foster

At the end, they had a drawing for some cool gadgets. I didn’t win anything, but that’s okay. It was a fun day!

DevFestMD 2017

Autopsy in Xubuntu 17.10

Well, I’m happy to report that I upgraded to Xubuntu 17.10 and Autopsy still works. That’s not surprising (it’s just Java, after all), but it still might be worth mentioning.

And I managed to process some data. At first, I made a dd image of a thumb drive because Autopsy couldn’t access it directly. That worked. It was my Strawberry Perl stick and Autopsy found over 6000 email addresses (presumably all of the module authors and other contributors to Strawberry Perl), but nothing else of interest.

Next, I grabbed a random thumb drive from my backpack and ran Autopsy as root. That way it could process the thumb drive directly, without making an image first. That drive contained a Linux driver, a zip file of all the slides from last year’s Enfuse conference, a tarball containing an older version of dd_rescue, an /etc/hosts file from my home network, a directory of photos from a trip to Ottawa, a PDF of a boarding pass for a plane trip, and an empty directory.

$ ls /media/tim/oylenshpeegul
agere_sta_fw.bin
_ALL SPEAKER PRESENTATIONS IN PDF.zip
dd_rescue-1.23.tar.gz
hosts
Ottawa 2013
Southwest_Airlines_-_Print_Boarding_Passes_and_Security_Document.pdf
System Volume Information

Autopsy found 110 email addresses from the zip file and one from the tarball. It found the EXIF data from the photos. The timeline showed a history for dd_rescue going back to the year 2000. It was easy to generate an HTML report

screenshot of Autopsy HTML report

and there were a half dozen other report formats as well. All in all, a pretty good experience! Now, how to write a plugin…

Autopsy in Xubuntu 17.10

Autopsy 4.5 works in Linux!

Ever since Autopsy 3.0 was released and right up through Autopsy 4.4.1, it only worked in Windows. I could get it to compile without error on my Linux machine, but running it would only display the splash screen of the little doggie and then immediately crash.

Screen shot of Autopsy 4.4.1 splash screen in Linux

But yesterday at OSDFCon, my friend Ben said he got the newly released Autopsy 4.5 to compile and run on his Ubuntu 16.04 machine! The weird part was that he said he didn’t do anything different, it just worked.

Today, I tried compiling it on my Xubuntu 17.04 machine and it did indeed work! It wasn’t completely straightforward, so here’s what I did:

First, get Sleuthkit 4.5…

$ cd ~/sleuthkit
$ tar xf ~/Downloads/sleuthkit-sleuthkit-4.5.0.tar.gz 
$ cd sleuthkit-sleuthkit-4.5.0

…and build it. It didn’t have a configure file included, so we’ll need need to run autoconf ourselves.

$ autoreconf --install
$ autoconf
$ ./configure
$ make
$ make check
$ sudo make install

Now we have to build the Java bindings for it.

$ cd bindings/java
$ ant dist-PostgreSQL

But Autopsy will be looking for Tsk_DataModel_PostgreSQL.jar rather than Tsk_DataModel.jar, so we’ll fake one of those.

$ cd dist
$ ln -s Tsk_DataModel.jar Tsk_DataModel_PostgreSQL.jar

Okay, now we’re ready to get and build Autopsy 4.5.

$ cd ~/sleuthkit
$ tar xf ../Downloads/autopsy-autopsy-4.5.0.tar.gz
$ cd autopsy-autopsy-4.5.0
$ export TSK_HOME=$HOME/sleuthkit/sleuthkit-sleuthkit-4.5.0
$ ant

Now when we run it…

$ ant run
...
BUILD SUCCESSFUL
Total time: 19 minutes 25 seconds

…we eventually get an Autopsy window!

Screen shot of Autopsy 4.5 in Linux

We can try to create a new case…

Screen shot of Autopsy 4.5 create case

…and process it.

Screen shot of Autopsy 4.5 processing

Hrm. The default settings apparently include things that only work in Windows.

So it’s still pretty aggressively single-platform, but at least it runs without crashing! To be fair, they describe it as “a Windows-based desktop digital forensics tool” right on the tin. But it’s written in Java, so there’s no good reason that it shouldn’t just work everyhere.

Assuming I can get it to actually process some data for me, next I want to try writing a plug-in for it in Clojure. I bounced that idea off of Brian Carrier and Richard Cordovano at OSDFCon yesterday. Both seemed skeptical.

Autopsy 4.5 works in Linux!