I am taking Data Science Ethics at edx.org. It has videos of the lectures and if you want you can look at the automatic transcription of the audio. I happened to notice that when he mentioned the tragedy of the commons, it transcribed “the tragedy of the comets.”
I don’t have a joke about this, I just find it really funny. I was reminded of @garybernhardt’s recent tweet about automatic form filling.
Should we really be worried about AI taking over everything?
I just finished taking “Applied Category Theory” at MIT! No, of course I didn’t get admitted to MIT; I watched it all online for free! The book (not yet published) is available online for free as well! What an amazing resource! Thanks so much to David Spivak and Brendan Fong for their incredible generosity. I guess thanks should also go to the Massachusetts Institute of Technology and to Cambridge University Press for allowing this to happen.
I loved this off-hand remark about the set of all sets that Spivak made in chapter 3, lecture 1, at 36:58.
“If you’re worried about that, it’s because you’ve been trained to be, probably, more than because it’s actually a worry.” – @david_i_spivak
Gosh… MIT, Stanford, UT Austin…what other schools that I could never get into can I attend on the internet?
I just completed weeks 0 and 1 of Introduction to Functional Programming in OCaml and I’m loving it! It took me way too long to figure out how scoping works in the “Definitions” section, but I think I finally got it. I watched the video over and over a bunch of times and kept trying examples in the REPL. It’s not C-like at all, but I guess it kind of makes sense.
The course is entirely in the browser, so there’s no need to install OCaml locally.
That’s great for the course instructors, who get to avoid the inevitable questions from the students with, er, proprietary operating systems. But it’s no way to live. I want to write code in my editor of choice and run it locally, so I went ahead and installed OCaml.
On this Ubuntu 18.04 machine, I installed OPAM, the OCaml Package Manager (like CPAN for Perl or cargo for Rust), with
sudo apt install opam
This installed OCaml
$ ocaml --version The OCaml toplevel, version 4.05.0 $ opam --version 1.2.2
but also installed the package manager in case I want to install any other modules as the course goes on.
Now, what’s that “toplevel” all about in that ocaml version line? In week 0, they explain that toplevel is what they call the REPL (read evaluate print loop) in OCaml. Now that we’ve installed it, we can invoke it with just “ocaml” on the command line. However, it doesn’t have readline support (boo!), so we need to invoke it with rlwrap
$ rlwrap ocaml OCaml version 4.05.0 # let foo = "Hello, World!";; val foo : string = "Hello, World!"
That’s enough for now. I’m sure I’ll be moving to my editor soon, but this is already better than working in the browser.
Last month, Julia 1.0 was released– hooray! I installed it on my Ubuntu 18.04 machine with
cd ~/Downloads gpg --import juliareleases.asc gpg --verify julia-1.0.0-linux-x86_64.tar.gz.asc cd ~/julia tar xf ~/Downloads/julia-1.0.0-linux-x86_64.tar.gz cd ~/bin ln -s ../julia/julia-1.0.0/bin/julia
That last part linked it to by bin directory, which is already in my path.
$ which julia /home/tim/bin/julia $ julia --version julia version 1.0.0
Today, I tried to use Julia from a Jupyter notebook. Jupyter is already installed.
$ which jupyter /usr/bin/jupyter $ jupyter --version 4.4.0
I installed it last year with
sudo apt install jupyter-core jupyter-notebook
This has been working fine and I have notebooks in Python 2, Python 3, Perl 5, Perl 6, and Clojure. However, Julia didn’t like it.
$ julia
julia> using Pkg
julia> Pkg.add("IJulia")
...
julia> using IJulia
[ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
ERROR: LoadError: IJulia not properly installed. Please run Pkg.build("IJulia")
Stacktrace:
...
Running Pkg.build and a little bit of googling revealed that I didn’t have a jupyter-kernelspec command. Why not?
$ jupyter-kernelspec Command 'jupyter-kernelspec' not found, but can be installed with: sudo apt install jupyter-client
Ah. I hadn’t installed jupyter-client, just jupyter-core and jupyter-notebook. Running that command and then rebuilding IJulia worked a treat. Now, either
julia> using IJulia julia> notebook()
from the Julia prompt, or
$ jupyter notebook
from the shell prompt starts up Jupyter (and Julia 1.0 is now on the list of kernels from which to choose). I prefer the latter, since I change to my Jupyter directory first, which shows all of my notebooks. Starting it within Julia shows my entire home directory.
Anyway, the bottom line is that the Julia 1.0 executable seems to work great with the Jupyter provided by Ubuntu; we just have to install the client too!
sudo apt install jupyter-core jupyter-notebook jupyter-client
I just came across this talk on category theory by Eugenia Cheng. I thought the whole thing was pretty great, but I particulary liked this impromptu remark about the nature of mathematics:
A lot of maths is about how we can take individual objects and build them into new ones that are a bit more interesting. The converse of that is that if we can then understand big ones, relative to small ones, and break them down into small ones, then we don’t have to do as much work. Because then all we have to do is understand some small objects and building processes. And then we can understand anything. That’s how all of maths works. That’s why we like understanding prime factorisation, so that we can just understand prime numbers and then build them into big numbers. And why we like breaking topological spaces into small pieces and then building them all up out of circles and disks. And why we like expressing all groups as fundamental finite ones and then seeing how we can build everything else with that. It’s so that we don’t ever have to deal with complicated things. We can just think about simple ones and building processes.
It reminds me of what Rich Hickey said about design:
So what do I mean when I say good design? I think one of the most interesting things about design is that, you know, people think it’s generating this intricate plan. But designing is not that. Designing is fundamentally about taking things apart. It’s about taking things apart in such a way that they can be put back together. If that makes sense. So separating things into things that can be composed. That’s what design is. Right? If you just make this intricate thing, you really haven’t designed in a way that needs to support the things that designs need to support. Like change. Every component of a design should be kind of about one or very few things. That’s sort of the nature of it, breaking things down until they are nearly atomic. And only then do you take those things that you have broken apart and compose them to solve the problem you set out to solve. But the first job is to take things apart in such a way that you can do that. And a good design process is iterative. This isn’t a grand plan you do once and then you go and finish.
Perhaps software design isn’t as far from category theory as Hickey seems to think!
A couple of months ago, I mentioned the CoRecursive podcast with Jim Blandy. Well, I just finished listening to part two and it was terrific! Like Adam Bell, I am still reading the book and it’s nice to hear from an author again now that I’m further along. I guess I am a “hanger-on” in the world’s smallest book club!
I’m currently reading Programming Rust by Jim Blandy and Jason Orendorff. I’m really enjoying it.
I bought it after hearing this talk by Orendorff. I loved the talk, so I bought the book. I figured if he can put together a talk that good, he can probably put together a pretty good book too. And I was right!
More recently, I listened to this podcast with Blandy. I really enjoyed this as well. So it’s no wonder I’m enjoying the book. I have no idea how they divided the writing duties, but it doesn’t matter— I like both styles.
Last year, I got Autopsy 4.5 to work in Linux, including processing some data. Now, Autopsy 4.7 is out and it claims even better Linux support. Today, I decided to try it.
First, I’ll uninstall the existing sleuthkit.
$ cd ~/sleuthkit/sleuthkit-sleuthkit-4.5.0 $ sudo make uninstall ...
Now, I’ll follow the instructions for Linux
$ sudo apt install libvhdi1 libvmdk1 libvhdi-dev libvmdk-dev libpostgresql-jdbc-java libc3p0-java ... $ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 $ sudo dpkg -i ~/Downloads/sleuthkit-java_4.6.1-1_amd64.deb ... $ cd .. $ unzip -l ~/Downloads/autopsy-4.7.0.zip $ cd autopsy-4.7.0/bin $ ./autopsy $ chmod +x autopsy $ ./autopsy WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.netbeans.ProxyURLStreamHandlerFactory (file:/home/tim/Autopsy/autopsy-4.7.0/platform/lib/boot.jar) to field java.net.URL.handler WARNING: Please consider reporting this to the maintainers of org.netbeans.ProxyURLStreamHandlerFactory WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Library not found in jar (libtsk_jni) SleuthkitJNI: failed to load libtsk_jni
Hrm. It didn’t work. The graphical window says this
org.netbeans.InvalidException: StandardModule:org.sleuthkit.autopsy.core jarFile: /home/tim/Autopsy/autopsy-4.7.0/autopsy/modules/org-sleuthkit-autopsy-core.jar: java.lang.UnsatisfiedLinkError: org.sleuthkit.datamodel.SleuthkitJNI.getVersionNat()Ljava/lang/String;
So where is libtsk_jni?
$ dpkg -l | grep sleuth ii sleuthkit-java 4.6.1-1 amd64 tools for forensics analysis on volume and filesystem data $ dpkg --contents sleuthkit-java_4.6.1-1_amd64.deb drwxr-xr-x root/root 0 2018-05-08 11:38 ./ drwxr-xr-x root/root 0 2018-05-08 11:38 ./usr/ drwxr-xr-x root/root 0 2018-05-08 11:38 ./usr/lib/ drwxr-xr-x root/root 0 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/ -rw-r--r-- root/root 935224 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk.so.13.4.2 -rw-r--r-- root/root 75042 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk_jni.a -rw-r--r-- root/root 1719320 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk.a -rw-r--r-- root/root 133480 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk_jni.so.0.0.0 -rw-r--r-- root/root 1174 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk.la -rw-r--r-- root/root 1225 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk_jni.la drwxr-xr-x root/root 0 2018-05-08 11:38 ./usr/share/ drwxr-xr-x root/root 0 2018-05-08 11:38 ./usr/share/java/ -rw-r--r-- root/root 5131696 2018-05-08 11:38 ./usr/share/java/sqlite-jdbc-3.8.11.jar -rw-r--r-- root/root 1399359 2018-05-08 11:38 ./usr/share/java/sleuthkit-4.6.1.jar drwxr-xr-x root/root 0 2018-05-08 11:38 ./usr/share/doc/ drwxr-xr-x root/root 0 2018-05-08 11:38 ./usr/share/doc/sleuthkit-java/ -rw-r--r-- root/root 512 2018-05-08 11:32 ./usr/share/doc/sleuthkit-java/copyright -rw-r--r-- root/root 196 2018-05-08 11:33 ./usr/share/doc/sleuthkit-java/changelog.Debian.gz lrwxrwxrwx root/root 0 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk.so.13 -> libtsk.so.13.4.2 lrwxrwxrwx root/root 0 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk_jni.so -> libtsk_jni.so.0.0.0 lrwxrwxrwx root/root 0 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk.so -> libtsk.so.13.4.2 lrwxrwxrwx root/root 0 2018-05-08 11:38 ./usr/lib/x86_64-linux-gnu/libtsk_jni.so.0 -> libtsk_jni.so.0.0.0 $ ls -l /usr/lib/x86_64-linux-gnu/libtsk* -rw-r--r-- 1 root root 1719320 May 8 11:38 /usr/lib/x86_64-linux-gnu/libtsk.a -rw-r--r-- 1 root root 75042 May 8 11:38 /usr/lib/x86_64-linux-gnu/libtsk_jni.a -rw-r--r-- 1 root root 1225 May 8 11:38 /usr/lib/x86_64-linux-gnu/libtsk_jni.la lrwxrwxrwx 1 root root 19 May 8 11:38 /usr/lib/x86_64-linux-gnu/libtsk_jni.so -> libtsk_jni.so.0.0.0 lrwxrwxrwx 1 root root 19 May 8 11:38 /usr/lib/x86_64-linux-gnu/libtsk_jni.so.0 -> libtsk_jni.so.0.0.0 -rw-r--r-- 1 root root 133480 May 8 11:38 /usr/lib/x86_64-linux-gnu/libtsk_jni.so.0.0.0 -rw-r--r-- 1 root root 1174 May 8 11:38 /usr/lib/x86_64-linux-gnu/libtsk.la lrwxrwxrwx 1 root root 16 May 8 11:38 /usr/lib/x86_64-linux-gnu/libtsk.so -> libtsk.so.13.4.2 lrwxrwxrwx 1 root root 16 May 8 11:38 /usr/lib/x86_64-linux-gnu/libtsk.so.13 -> libtsk.so.13.4.2 -rw-r--r-- 1 root root 935224 May 8 11:38 /usr/lib/x86_64-linux-gnu/libtsk.so.13.4.2
That seems right…why can’t it find it? Oh, well; forget the package…
sudo dpkg -r sleuthkit-java sudo dpkg --purge sleuthkit-java
Let’s try building the entire sleuthkit from source
$ cd ~/sleuthkit $ tar xf ~/Downloads/sleuthkit-4.6.1.tar.gz $ cd sleuthkit-4.6.1 # Hey, there's a configure script already! $ ./configure $ make $ make check $ sudo make install # All set? $ which mmls /usr/local/bin/mmls $ ls bindings/java/dist/ sleuthkit-4.6.1.jar $ cd ~/sleuthkit $ unzip ~/Downloads/autopsy-4.7.0.zip $ cd autopsy-4.7.0/ $ bash unix_setup.sh photorec found Java found in /usr/lib/jvm/java-8-openjdk-amd64 /usr/local/share/java/sleuthkit-4.6.1.jar found Copying into the Autopsy directory Autopsy is now configured. You can execute bin/autopsy to start it $ bin/autopsy ... SleuthkitJNI: loaded libtsk_jni ...
Success! Aw, but when I started a new case, it crashed!
The pop-up window said
Sorry, the application java has stopped unexpectedly.
If you notice further problems, try restarting the computer.
and the terminal said
# A fatal error has been detected by the Java Runtime Environment: ... # Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %d %P" (or dumping to /home/tim/core.21205) # # An error report file with more information is saved as: # /home/tim/hs_err_pid21205.log ... # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. ...
So. I don’t really know what to do about this setback. This Ubuntu machine is now at 18.04, but I doubt that’s the problem. I’m using OpenJDK, not Oracle Java…but if that’s the issue, then I don’t want to play.
In February, Mary Sheeran gave a talk at LambdaDays called “In praise of Higher Order Functions and of some friends and heroes”. I was not able to attend LambdaDays, but I watched the video of the talk and I really enjoyed it.
Sheeran is urging us to read these historic papers, but I was disappointed to be unable to find many of them on the internet. It wasn’t until her 41st slide that she referred to something that we can see unimpeded! I guess it’s not really possible for an amateur to read all of these papers. One really must belong to a university or some other institution that subscribes to these journals and has (or can borrow) these books. What a shame.
Sheeran ended her talk with an admonishment from Christopher Strachey to bring together theory and practice.
It has long been my personal view that the separation of practical and theoretical work is artificial and injurious. Much of the practical work done in computing, both in software and in hardware design, is unsound and clumsy because the people who do it have not any clear understanding of the fundamental design principles of their work. Most of the abstract mathematical and theoretical work is sterile because it has no point of contact with real computing.
Strachey said that over fifty years ago in another paper I can’t find on the internet (“Towards a formal semantics”, 1966), but it doesn’t seem like we’ve listened. I’m a working programmer and I can’t easily read this research.
Anyway, I did track down some of the papers, so I thought I’d post the links I found. Some of the links are PDFs.
Yesterday, I drove up to Lancaster for the Central Pennsylvania Open Source Conference. It was terrific! This is my third time doing this and it’s become one of my favorite conferences. There are three tracks of talks most of the day, so you can almost always find something interesting to learn about at every session. Often, you have to decide between two or more talks that you’d really like to see.
Don Schenck had lots of practical advice for beginners in a talk called “OMG How Do I Even Start An Open Source Project?”
Matt Rogish talked about “Kubernetes 101” and how to pronounce kubectl (“coob see tee el” or “coo bee cuttle”).
In “Kubernetes: A Deep Dive,” Mike Newswanger pronounced kubectl “cube control.”
Rogish and Newswanger both used Docker for all of their examples. They both mentioned that you could run Kubernetes without Docker, but said no more about it. I was kind of hoping to hear about using Kubernetes to orchestrate other things.
Jason O’Donnell talked about “Vault: A Tool for Managing Secrets” and had a nice demo.
James Tramel did an “Introduction to Blockchain.” This was nearly impossible to do in thirty minutes. He did a fine job, but there were clearly more questions in the room when we had to stop.
Sarah Mogin gave a really great talk called “Building for Performance and Scale with React/Redux.” I don’t usually have much patience for JavaScript, but she had ton of practical advice with no sugar coating.
“Man, JavaScript is annoying.” – @sarahmogin
Much of her talk had an A versus B structure for a variety of A’s and B’s (traditional versus headless, Component versus PureComponent, &c.) which was very effective.
I also really enjoyed Steve High‘s talk “Ditching REST for gRPC.” His demo was an arithmetic service “because who doesn’t like math.” He live-coded subtract into the service during the talk. He didn’t hear me answer “minuend” when he asked, so his variables were called firstnumber and subtrahend. Naming things is hard.
There were also lightning talks, 3D printing by PrintedSolid.com, and lots of great local food.
The folks from MajorMega had a prototype of Hyperland set up that we could try out. In addition to VR headset and controllers, it had fans for wind and a floor that tipped. This guy thinks he is riding on the back of a moving truck and shooting at other cars and things chasing him.
There were also giveaways, including a 3D Printer from LulzBot. I didn’t win anything, but that’s okay. It was fantastic day!
Oylenshpeegul 