If you take an interest in why — not how — the modern large language models work, this is a great 30-minute video lecture from Bertrand Serlet, former Senior Vice President of Software Engineering at Apple. This is a well-crafted, non-jargon talk to take in; at the very least, it’s a lesson in how to communicate a complex, mathematical topic in incremental, bite-size pieces without devolving into multi-layer flowcharts and equations.

I started watching because I remember Serlet from a couple of his hilarious presentations from his Apple days (watch here, then here) poking fun at Microsoft, and I’m glad I didn’t skip this one.

If, after watching the video, you would like to follow up on the topic with some technical reading, I have you covered. Transformers are a type of deep neural network, specifically trained to make predictions based on chains of previous input, using the inherent contexts and relationships therein.

So, while an image classifier takes in a single image and predicts probabilities that the image contains specific objects, a transformer takes in a sequence of information pieces, chained together, and makes a contextual prediction. The prediction in this case tries to extend the input chain, which can be the next logical word, next logical pixel, next logical musical note, etc.

If you want to take a deep dive into building a simple LLM from scratch, you may start with Andrej Karpathy’s tutorial. Karpathy is one of the co-founders of OpenAI (makers of ChatGPT), and this is a very well put-together lecture. He also has an hour-long “busy-person’s intro to LLMs”.

Finally, if you really want to go into the rabbit-hole, this paper is what started the transformer revolution: Attention is all you need. The View PDF link will take you to the PDF version of the paper. This LitMaps link will show you how influencial that paper has been.

But, seriously, forget all of the technical stuff. Go watch Bertrand’s video lecture.

P.S.: I should note, this is not an unconditional endorsement of AI in general and LLMs in particular. These technologies are being used, and may continue to be used, in ways that are unsavory, short-sighted and dangerous. We need to be circumspect and judicious in how we deploy these extremely powerful technologies, so that we don’t incur unacceptable costs in the long term. We should aim to bolster our creative crafts with AI, not foolishly attempt to replace human creativity.

The Guardian reports:

The collection, titled Marple, marks the first time anyone other than [Agatha] Christie has written “official” (as recognised by the Christie estate) Miss Marple stories. The 12 women who contributed to the collection include award-winning crime writers Val McDermid and Dreda Say Mitchell, historical novelist Kate Mosse, classicist and writer Natalie Haynes and New York Times bestselling author Lucy Foley.

(If the Guardian link above doesn’t work for any reason, here is an alternative link from Smithsonian Magazine quoting The Guardian.)

This is great! Always room for more Marple mysteries for avid Christie readers such as me!

There have already been several new “official” Poirot novels, written by Sophie Hannah, also sanctioned by the Christie estate, that have been published in the last few years. I have read a couple of them, and they are pretty good reads! The author’s voice seems just that bit different — of course, that is to be expected, and indeed hoped for — and that’s a little jarring after years of reading Christie, but the plots and the characters are quite well-thought-written-fleshed-out. They won’t feel out of place amongst Christie’s Poirot mysteries.

If these new Marple stories are anywhere as good, then they will be worth looking out for.

Here are some news snippets from the last few weeks.

• Yangtze shrinks as China’s drought disrupts industry (NPR)
• Drought Causes Yangtze – China’s Most Important River – To Dry Up (scitechdaily.com)

As China’s most important river, the Yangtze provides water to more than 400 million Chinese people. This summer, with rainfall in the Yangtze basin around 45% lower than normal, it reached record-low water levels with entire sections and dozens of tributaries drying up. The loss of water flow to China’s extensive hydropower system has created problems in Sichuan, which receives more than 80% of its energy from hydropower.

• Pakistan Flood Situation To Worsen As Indus And Swat To Swell Further, Monsoon Fury Claims Nearly 1,000 Lives (cnbctv18.com)
• Pakistan floods leave more than 1,000 dead, a half million survivors in camps (wtoc.com)

Nearly a half million people crowded into camps after losing their homes in widespread flooding and the climate minister warned Monday that Pakistan is on the “front line” of the world’s climate crisis after unprecedented monsoon rains wracked the country since mid-June, killing more than 1,130 people.

• Florida’s latest homeowners insurance crisis is a mess decades in the making (tampabay.com)

The drama is just the latest problem as the state experiences its biggest insurance crisis since Hurricane Andrew in 1992. […] In the last two years, more than 400,000 Floridians have had their policies dropped or nonrenewed. Fourteen companies have stopped writing new policies in Florida. Five have gone belly-up in 2022 alone. The record, set after Hurricane Andrew’s devastation, is eight in one year.

The latest casualty was Coral Gables-based Weston Property & Casualty, which leaves 22,000 policyholders — about 9,400 in South Florida — scrambling to find new insurance companies.

Costs also have skyrocketed. In 2019, when DeSantis was sworn in, Floridians paid an average premium of $1,988. This year, it’s now $4,231, triple the national average, according to an Insurance Information Institute analysis.

• Major sea-level rise caused by melting of Greenland ice cap is ‘now inevitable’ (The Guardian)

[T]he study published in the journal Nature Climate Change used satellite measurements of ice losses from Greenland and the shape of the ice cap from 2000-19. This data enabled the scientists to calculate how far global heating to date has pushed the ice sheet from an equilibrium where snowfall matches the ice lost. This allowed the calculation of how much more ice must be lost in order to regain stability.

The research shows the global heating to date will cause an absolute minimum sea-level rise of 27cm (10.6in) from Greenland alone as 110tn tonnes of ice melt. With continued carbon emissions, the melting of other ice caps and thermal expansion of the ocean, a multi-metre sea-level rise appears likely.

“It is a very conservative rock-bottom minimum,” said Prof Jason Box from the National Geological Survey of Denmark and Greenland (Geus), who led the research. “Realistically, we will see this figure more than double within this century.”

Climate change is happening, our civilization as it currently stands will be upended because of it, and we as a global society have done (next to) nothing to mitigate it. The best time to take measures to decelerate climate change was decades ago; the next best time is right now. Either we grit our teeth and hold our breath through a couple of decades of accelerated, painful, transition to sustainable energy use, or… we will be forced to hold our breath under water as our coastal life submerges.

By the way, 40% of the world’s population lives within 100km (60mi) the coast.

This is such a beautiful song; if you haven’t heard it, here’s a version on Youtube! (There are a couple other versions, such as this one, also great.)

Hat tip to Poorna for making me listen to this on one of our uncountable night drives; it has since been on repeat play for me.

—

Song: Nao
Lyrics: Ritam Sen
Music: Prasen
Group: Hoodkhola Kobitara

ekhon nistobdho mohonaye
eshe dariyeche dosh-jon shundor
bati ghorer naw-sho janalaye
koto pakhi khujche mrityur uttor!

ekhon nistobdho mohonaye
eshe dariyeche dosh-jon shundor
bati ghorer naw-sho janalaye
koto pakhi khujche mrityur uttor!

jeno churi jawa ek phali bhor aaj
mridu chhuye achhe himel gallery
jeno churi jawa ek phali bhor aaj
mridu chhuye achhe himel gallery

ei, ei ei ei
ei bhor nao, bondor nao
nao ey-nistobdho mohonao
nao ey-nistobdho mohonao

mm-hm ei, ei ei ei
ei bhor nao, bondor nao
nao ey-nistobdho mohonao
nao ey-nistobdho mohonao

—

shudhu tumi, ar tumi, ar tumi
koto mrito potrikaye kartuj-e
golaper sugondhi guhaye
aw-prem er ondor e chokh buje

shudhu tumi, ar tumi, ar tumi
koto mrito potrikaye kartuj-e
golaper sugondhi guhaye
aw-prem er ondor e chokh buje

aaj chand-er ghor makhto bichana
tomar podo-dhhoni lukoye bali te
aaj chand-er ghor makhto bichana
tomar podo-dhhoni lukoye bali te

ei, ei ei ei
ei duur nao, roddur nao
nao ey-nistobdho mohonao
nao ey-nistobdho mohonao

mm-hm ei, ei ei ei
ei bhor nao, bondor nao
nao ey-nistobdho mohonao
nao ey-nistobdho mohonao

nao ey-nistobdho mohonao
nao ey-nistobdho mohonao

A Silk Road stopover might have been the epicentre of one of humanity’s most destructive pandemics.

People who died in a fourteenth-century outbreak in what is now Kyrgyzstan were killed by strains of the plague-causing bacterium Yersinia pestis that gave rise to the pathogens responsible several years later for the Black Death, shows a study of ancient genomes.

“It is like finding the place where all the strains come together, like with coronavirus where we have Alpha, Delta, Omicron all coming from this strain in Wuhan,” says Johannes Krause, a palaeogeneticist at the Max Planck Institute for Evolutionary Anthropology in Leipzig, Germany, who co-led the study, published on 15 June in Nature.

Fascinating read on new research on the origins of Black Death. As you can imagine, it’s not an easy task to find genomic data from the plague bacteria, several centuries after the pandemic. Then, like now, how the pandemic spread mattered quite a lot of how and where a lot of humans came together and then dispersed, carrying the deadly disease with them.