(Note that there’s now a Telegram group that you can join to be kept up to date with these posts. I’m never going to make the A-List, but at least I haz the gimmicks!)
This edition of the weekly (haha) head voices attempts to reflect on the period of time from Monday November 5 to Sunday November 18, 2018.
The following action scene happened exactly halfway through:
Running aka Irony update
Seeing that you’ve made me talk about running again, have a look at this photo of one Luna Mono 2.0 after about 700 km of (mostly road) running in about seven months, and one brand new Luna Mono 2.0:
At around about the same time as the new shoes arrived, shortly after South African customs charged me a painful amount before letting the new babies through, both my ankles, from around the posterior tibial tendon area, let me know in no uncertain terms that they were now demanding a break.
After repeated explanations by my life partner (she counts being a rheumatologist amongst her many talents), and by a foot surgeon friend, that my flat feet mean that my posterior tibial tendons have to work even harder than they would usually have done had I been anatomically speaking more normal, I had to start facing the music:
I was going to have to wear normal person running shoes again.
(If I have to be honest I would have to say that the music was in fact more about having to take a running break. I had sneakily been pushing up my weekly distance, trying to run through ankle discomfort, and this was probably the true core of the problem.
All of that being said, I am choosing to interpret matters a bit differently. Running breaks are really hard yo.)
I’ve now done two runs in my pre-Mono Kinvara 8s, and it does indeed feel (of course it does) like my ankles might slowly be recovering. I am hopeful that the trend continues, and that I can eventually rotate in my Lunas again.
Nerd toys update: RTX 2070 in da house.
After weeks of deliberating, I broke down and bought an NVIDIA RTX 2070 for deep learning.
This in turn led to a flurry of experimentation and to be quite honest a slight case of deep learning binging.
At least I have the following new blog posts to show for it:
(I know that some of these occurred outside of the two week timespan covered by this post.)
On the memory saving of mixed-precision training.
In my tests with ResNet50, a serious convolutional neural network for image classification, the exact same network with the exact same training settings required 14159 MiB in fp32 mode but only 7641 MiB in mixed precision mode.
This means that in some cases, this new RTX 2070 can go toe-to-toe with many far more expensive cards.
Furthermore, I informally measured a training speed boost of about 20% with the smaller ResNet34.
… in test screenings, Willy Wonka had a scene with a hiker seeking a guru, asking him the meaning of life. The guru requests a Wonka Bar. Finding no golden ticket, he says, “Life is a disappointment.” The director loved it, but few laughed. A psychologist told him that the message was too real.
It also says I’ve done 27km in my Xero Genesis sandals, or as I have begun to call them, Xero Tolerance.
You make one mistake, and something will break. You do get to keep all the bloody pieces.
In any case, when I started on this barefoot-style / natural running adventure, I had subconsciously set myself the limit of 200km before evaluating the success of the experiment.
At 200km, the experiment was still unsuccessful (different parts of feet and ankles were taking turns complaining) so I moved the threshold to 300km, with the plan to move it to 400km if required.
I call this The Stubborn Scientific Method(tm): You keep running the experiment (harr harr) until it says what you want it to say.
To be fair, in this specific case an injury would have (and still can), stop the experiment. Most fortunately the muscles, bones and tendons in my feet, ankles and calves, although complaining quite audibly, have held up.
This past Sunday I did a long(ish) run where it felt for the first time like my feet and ankles had finally toughened up enough (and perhaps my form had also improved slightly) to just keep on propelling me forward quietly and efficiently.
Together with the brilliant sunny winter morning conditions, this conspired to reconfigure my face machine into a rather long-lasting grin.
I am carefully optimistic that I might be able to make this specific adventure a more permanent one, and that makes me really happy.
The Emacs Section
NERD-ALERT. SKIP TO THE NEXT SECTION IF YOU ARE NOT INTO TEXT EDITORS!
A friend from work sent me a ZIP file with research data.
I was super surprised that I could easily decompress the ZIP file using Emacs Dired (Dired is of course the file-manager built into Emacs, doh), but that there was no easy way to mark and extract specific files from the archive.
It worked, but it didn’t default to the opposite Dired file-list pane as all commander-style tools should do, and by default it re-created relative paths, which is the opposite of the default in most two-pane commanders I know.
As is the wont of Emacs users, I reshaped the code ever so slightly to work like I thought it should.
Shaping Emacs Lisp code has a pleasant fluid feeling to it. Code is data, code is configuration, data flows through code.
I’m telling you this story, because it was a nice little reminder of one of the reasons I like this software so much.
Differentiable Image Parameterizations, a beautiful machine learning article on Distill that surveys and showcases different techniques for generating beautiful images with deep learning. These networks sort of learn to see in order to solve specific tasks, but you can tickle them in different ways to get them to show you the insides of their visual circuitry, and it’s quite beautiful.
The Prophylactic Extraction of Third Molars: A Public Health Hazard is an article which was published all the way back in 2007. It makes the claim that at least two thirds of wisdom tooth extraction are unnecessary. One could say that their only function is to… extract your money. BA DUM TSSSSS! To that I would like to add: WHY DENTISTRY WHY? HAVE YOU NOT HURT US ENOUGH?!
A colleague at work emailed this TechCrunch post about a 3D printed neural network that diffracts light going through in order to do its trained inference work on incoming images. Although it’s a retro-futuro-mind-bending idea to do it with a whole neural network, and it smacks of hell-yeah-this-is-what-scifi-promised-me-that-AI-would-look-like, I could not help but recall a certain Very Flat Cat telling us about this sort of passive light-based computation almost 20 years ago.
The Poetry Section
GOU#1 had to select an English poem to recite for class.
I had forgotten how much subtlety and recognisable human complexity this poem was able to pack into such a petite little frame. If you have the time, read the analysis linked above after spending some time with the poem itself.
Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;
Then took the other, as just as fair,
And having perhaps the better claim,
Because it was grassy and wanted wear;
Though as for that the passing there
Had worn them really about the same,
And both that morning equally lay
In leaves no step had trodden black.
Oh, I kept the first for another day!
Yet knowing how way leads on to way,
I doubted if I should ever come back.
I shall be telling this with a sigh
Somewhere ages and ages hence:
Two roads diverged in a wood, and I—
I took the one less traveled by,
And that has made all the difference.
Friends, no matter which paths you take this week, I hope that we may meet again.
This post covers the week from Monday July 9 to Sunday July 15.
The business part of my week was unfairly dominated by far too much after-work obsessing over programming languages, with which I seem to have an unhealthy (or perhaps not) obsession.
I will externalise some of these thoughts further down in this post.
I’m starting with a weekend / running update, which should be reasonably safe for non-nerds to read. However, after that, the nerd dial will go up to 11 with stuff about tools and programming languages right up to the end of the post.
I would have wanted to use the adjective “face-melting”, but I’m not sure if any intensity of nerdery could ever reach that level.
We can dream.
Weekend running update
Most fortunately the weekend had other plans and supplied us with at least 2.5 parties, the first of which even culminated in a ridiculously fun trail run in the mountains on the winter morning after.
The winter morning sun was just perfect, the company was great, and I had forgotten all forms of performance tracking devices at home.
Readers with bionic eyes might notice the Lunas on my feet.
I have now ran just over 260km in them, but, in a surprise twist to the regular readers of this blog, my biological equipment has still not yet completely adjusted to the new style of locomotion.
They currently have to work extra hard to stabilise my feet while running, because, you know, no shoes.
Because doing this thing was not hard enough already, and because the Lunas are perhaps still a bit too cushiony, and because my friend the Very Flat Cat forgot that I’m very suggestible after 11:00 in the morning when my prefrontal cortex takes the rest of the day off, I am now also the very shy owner of a pair of Xero Genesis running sandals:
The soles are only 5mm thick, and quite hard, being rated for a few thousand miles and all. The upshot of this is that one’s feet have to work even harder than in the Lunas.
My first run in these was amazing: I could feel my feet reacting to every little pebble, and my running style having to adapt even more to the terrain.
However, there was a price to pay for all of that additional terrain feel (and the fact that I took a much longer maiden run than I should have): The next day, the tendons in my feet felt even more (ab)used than usual.
WITH GREAT POWER COMES GREAT RESPONSIBILITY, it seems.
Due to these shoes being so powerful, I have had to resign to introducing Xero running far more gradually than I had initially thought.
Vacation-based-thinking-driven tool sharpening aka The WVV 2018 Data Science Toolbox(tm).
During the previously blogged-about Mpumalanga vacation, the lack of alarms, devices, and other work accoutrements, resulted in there being ample time for staring-into-space-grade thinking sessions.
During one of these thinking sessions, I realised that I had somehow neglected my data science toolbox for a while.
At some point a few years back, I was so into ipython notebooks (what has now become jupyter) that I used them as my main work lab notes modality.
However, in the meantime I had fallen slightly out of love with the computational notebook style of data programming, because I had begun to develop doubts about their role in the analysis pipeline.
interlude 1:jupyter notebooks are nice for initial data exploration, and they’re especially useful for remote computation with embedded graphics. However, that initial momentum of discovery risks devolving into an unwieldy monolith of code snippets, data transformations and experiments. There’s a fine line to be walked between flexible experimentation on the one hand, and version-controlled, time-stamped, permutational and scientific rigour on the other.
interlude 2:I have to apologise for using the term “data science” in a non-comedic context. In spite of the inherent humour, it has turned into a usable blanket term for computational data understanding.
Due to my growing doubts in the order of Jupyter, and due to being occupied with less traditionally data sciencey work projects, I had unfortunately let my data science toolbox gather perhaps a bit too much dust.
Slightly more worrying than falling out of love with the Jupyter Notebooks (I still like them, I’m just not that madly in love anymore), was the more specific issue that I’d even let the datavis parts get a bit dusty.
Although I should probably write a more complete post about this, here is the list of ingredients of the official 2018 WHV Data Science Toolbox(tm):
Programming language and library ecosystem: Python.
This language, in spite of its shortcomings, dominates the data science / machine learning world thanks to its STELLAR ecosystem.
numpy, pandas, scipy, scikit-*, tensorflow, pytorch, keras, cython… this snowball has turned into a pretty sizeable planet.
For this reason, it would be hard to justify any other choice for data science.
However, since I’ve been seeing more of Lisp and the rest of the ever-expanding programming language landscape, I can see (Python’s shortcomings as a programming language) clearly now.
In terms of interactive programming, Python beats the majority of practical programming languages, with Common Lisp being one notable exception.
However, it’s not functional enough, which engenders unnecessarily imperative, side-effecting code. More specifically, it’s not expression-oriented.
More about this slightly further down. Maybe.
Datavis: Anything, as long as it’s Vega or Vega-Lite.
I spent a few years of my life wrangling d3.js, down to INNARD-LEVEL.
Mike Bostock’s idea of data-element-joins is genius, and internalising it was intellectually satisfying.
(if it’s any consolation, the new kid can be considered the grand-child of d3.js.)
vega and vega-lite are so-called visualization grammars, or visualization DSLs (domain specific languages).
The upshot is that one codes up a chart, or a whole set of linked charts and their interactive behaviour, using a language that was designed for this purpose.
This chart code can be easily shared, or converted into interactive visual representations that can be embedded in applications, online or in print quality documents.
With Altair, you can even send your pandas dataframes to vega and vega-lite charts all from the comfort of your slightly defective Python armchair.
Development Environment: PyCharm.
You knew it was not going to be Jupyter Notebooks, but you probably expected it to be Emacs.
Well it’s not. Surprise!
The remote interpreter support in PyCharm enables me to connect to a Python virtual environment anywhere on the planet, which I often do.
The JetBrains wizards have optimised the remote communication of code intelligence, so completion, documentation and general code understanding is almost indistinguishable from that on a completely local project.
Two notable drawbacks are visualization and long-running jobs.
For the long-running jobs I do tend to use Jupyter Notebooks or when at all possible mosh, which is amazing. However, because the primary modality is not the notebook, my code is versioned and organised into separate libraries which I can call into from notebook or mosh.
For visualization, it’s either connecting to the altair chart server via SSH pipe, dumping the chart to the unison-synced project, and/or a Jupyter Notebook.
Of course you use Postgres on an SSD for your data, and of course you know enough SQL to make short work of most of the heavy-weight transformations often required at the start your data crunching pipeline.
For all of my lab notes, reports, books, papers and blog posts, I use Emacs Org mode.
LaTeX math with live preview, live code snippets, SVG graphics, bibtex references, export to anything. This is one of the best ways to document your science.
Programming language addiction update.
I spend far too much obsessing over programming languages, old and new.
For the past two weeks, I wasted even more precious time than usual reading up about programming languages.
Because I would really like to spend more of my time on other, perhaps more valuable activities, I’ve been trying to better define what it is I’m actually looking for.
Of course there is no single best programming language, but a whole set of good languages that map in intricate ways to different problem domains.
In spite of this, I have been pining for a language with, in order of importance:
A Functional Programming DNA, with which I’m referring to a) expression-orientedness, b) a preference for pure functions, and at a higher level, c) the modelling of reality as more or less explicit dataflows.
Interactive programming, with Common Lisp being the textbook example of this.
Great tooling and IDEs, meaning first-class support by something from JetBrains, Microsoft or Emacs.
Great concurrency and parallelism stories.
A great library ecosystem.
Modest memory use.
Having just explicitly written this down for the first time (!! – it was consuming so much glucose just being kept amorphously swirling around in my brain) I can now mentally map some of my most recent language dalliances to these points.
This language is far too simple for my taste, but probably really great for teams.
I did recently take a more serious look when setting up a telegram bot using tbot and being amazed at how simple it was building web services like these using goroutines and channels.
Go satisfies points 3 to 6 from the list above. Makes sense that I decided to file this experiment away under “check when you need to put a webservice together REALLY QUICKLY”.
When I saw up that rust, surprisingly, is an expression-oriented language, I flew through the O’Reilly Programming Rust book I had bought previously as part of a bundle.
Evaluating rust by the list above, we award it a fractional 1 because expression-oriented, 3 due to jetbrains plugin amongst others, 4(ish) – great memory safety, but compared to clojure, concurrency and parallelism stories still have much room to grow, a solid 5 thanks to cargo and a very strong 6.
I filed this one away under “re-evaluate whenever you reach for your trusty C++”. (also, actix-web looks amazing for super high performance microservices.)
We’re getting back on track with the WHVs friends!
In the hardly started tradition of writing blog posts in music-backed focus blocks, I have my “upbeat thinking” playlist teed up and ready to go. The outline of this post formed itself as a Real Bullet List(tm) in my Emacs about an hour ago.
They grow up so fast
Theory of Mind, or ToM, is an important mental capability that we use to model and predict the thoughts and desires of fellow humans.
Just the other day, as we were going through our school morning ritual of the offspring units eating breakfast together and the adults self-administering the correct number of espressos required for normal functioning, GOU#3 calmly informed me from her mother’s lap:
Daddy, mommy would like another biscuit with her coffee.
Genetic Offspring Unit #3 only very recently turned 2.
With this request, she demonstrated surprising levels of ToM and planning ability. She inferred, entirely correctly, what her parental unit required at that moment, and performed exactly the correct action (delegation, yikes!) to satisfy that requirement.
I am still suitably impressed.
Sketchnote your life
Sketchotes refer to a type of hand-written notes that employs both writing and drawing techniques. Here’s an example by Emacs guru and famous internet person Sacha Chua:
I’m trying to spend more time dedicated to thinking and so-called conceptualising. Sketchnotes seems like a good tool to use during these thinking sessions, so last week this formed the ideal excuse to go out and acquire a new large Moleskine with blank pages (I used to use Moleskines for all my note-taking before going digital), and a whole bunch of sketchnote-recommended pens (Pilot Hi-TecPoint 0.5 which I already had one of; Pilot G-TEC-C4 for super fine drawing, pen also turned out to have best handling of the lot despite its simplicity; Pentel Energel 0.7mm).
My first session was spent sketching out my current life landscape (thank you KvG for this tip years ago), including work, side-projects and a bunch of developing and potential opportunities, as well as the links between them.
I can report that drawing like this is a great trick to keep one’s attention glued to the page, and hence to the chosen focus, whilst at the same time maintaining sufficient mental distance to process the more substantial thoughts and all of their interactions.
Telegram has the public group chat market cornered
Besides the dubious future of WhatsApp’s privacy (Founder #1 Jan Koum is planning to leave while Founder #2 Brian Acton recently donated 50 million dollars to Signal), the WhatsApp web-app is more irritation than it’s worth. The fact that I have to keep my phone awake and connected to the network is a silly constraint which even the far more secure Signal desktop app does not require.
Anyways, I digress.
This section is about Telegram, another messaging app with dubious security that at least does not belong to Facebook.
Besides all of its stickers, animated gifs, and (non-)useful bots everywhere, Telegram has two additional features which are quiet compelling:
Although it requires a telephone number to be setup, you can configure a username which you can give out to people instead of your telephone number to have them contact you. This adds an extra layer of privacy which is sometimes useful.
This is ideal for easily starting special interest groups, and can be seen as a modern and mobile-first form of IRC. The mobile apps are generally really fast and full featured.
Anyways, on a lark we created one such group, called ZA Tech Light, for tech people (aka nerds) in ZA. If you are such a person, or you just like chatting with nerds in ZA, feel free to drop in at @zatechlight. Although primarily lark-based, this could be seen as a sort of splinter group of the much larger (because older) Slack group called ZA Tech.
Yesterday, I did my second 10km+ run in the Lunas, bringing total sandal running distance to 107km.
I am now back up to my pre-sandal standard running route distance.
That being said, my calf muscles are still complaining quite loudly after every run. The recovery perioud seems to be shortening however, and the calf muscle complaints are less convincing every time.
All of THAT being said, running barefoot- aka primitive-style feels amazing, so much so that although one does keep an eye on things, one does not perceive the above-mentioned muscle discomfort as an issue at all.
Furthermore, the patella strap I previously had to wear during running, to prevent knee pain, has been lying in my cupboard, unused for the past 107km.
The side-project dilemma
Most nerds I know have side-projects.
It’s how we learn new things and keep ourselves constructively entertained.
Up to now, I’ve usually chosen my side-projects not only on the basis of learning, but also based on their business potential. Some of them have indirectly led to revenue, partially through the business-relevance check, but so far never directly. That is, I’ve never brought a side-project to market.
This weekend I had an idea for a pretty obscure side-project. In terms of creativity and learning, and of passion and brain-fit, it scores highly, but in terms of direct business potential quite the opposite.
I’m probably at least going to start, because it’s too much fun not to.
What is your approach in situations like this? Do side-projects have to satisfy any kind of utility requirement? Which criteria do you use to select your next side-project?
Thanks for reading this far peeps! I hope you have a beautiful week, and that we might meet again at the end.
Matters have started heating up around here with the final preparations for a pretty intense project that is expected to launch in about a week’s time. I would prefer saying more about it after the launch, because reasons.
I’m mentioning it here as explanation for the paucity of this post, and just in case a near-future post manages to be slightly late or even absent.
All will become clear in due time!
Sandal Running Progress Report #1
During the past week, I sneaked in three more runs(shorter-than-usual, because don’t want to spoil it) in the new Luna sandals, after their maiden voyage last Sunday. Today’s run, which included the panoramic view at the top of this post, was up and down the mountain in hot and sunny Paarl.
My calves and feet are still getting used to the additional work, but they seem to be recovering more quickly after each run.
My running form has necessarily improved quite a bit. If you don’t run correctly in sandals, your feet and your legs let you know immediately.
(When you do run correctly, the pleasure is considerable.)
Scary snake #2
On Saturday, we had a pretty serious but fortunately bite-free encounter with a Cape Cobra.
Four of our kids (GOU #2 and #3, and one of their cousins) were playing inside the house, front door open.
We were outside, enjoying the balmy late afternoon weather.
GOU#1, who was also outside, calmly informed us that a snake was making its way in through the front door.
As I was still internally remarking the fact that it was a significant specimen of snake, probably about 1 metre, my partner fortunately sprang into action and shut the front door with the snake halfway through, temporarily trapping it.
I say temporarily, because that thing is powerful and it was probably in panic, and was managing to worm itself inside, to where the youngest of the kids stood still, transfixed by the snake-like motions of its head.
After a run-around searching for suitable implements, finally assisted by cousin #2, I had a spade and was able to remove the snake from the vicinity of the humans.
A sufficiently large snake induces a visceral fear reaction. The way they move and propel themselves over the earth seems to play a large part in that.
In South Africa, hospitals will treat these bites with a locally developed polyvalent antivenom that is effective against puff adder, gaboon adder, rinkhals, green mamba, Jameson’s mamba, black mamba, cape cobra, forest cobra, snouted cobra and Mozambique spitting cobra. Pretty hardcore.