Training nanoGPT on my Journal

Wouldn't it be great to have a pet AI chat bot that knows you really well, and can give you free consulting, coaching and therapy? What could possibly go wrong if an intelligent corporate-controlled system has unlimited access to your private data to provide you the best possible service? Well... a lot of things.

That's why I'm looking into building a helpful chat bot that runs 100% offline, purely on the CPU of my 7-year-old laptop.

I have megabytes of personal notes that I could use for training it, along with emails, chat logs, and other personal information that I would never entrust to an external entity. I will probably never get around reading all this stuff myself, but at last, those files have found a new life as delicious AI Crispy Bits :)

First I tried to run GPT-J-6B (see "Dying of a Deadstar") on my laptop as a starting point, but that doesn't seem feasible, with a whopping 24GB VRAM requirement (my laptop has less than 1GB). There are portable USB Tensor Processing Units (TPUs), such as Coral USB Accelerator, but it seems that they can only run "TensorFlow Lite" models with only 8-bit precision, that rules out GPT-J-6B without doing extra work.

Then Andrej Karpathy released his enlightening lecture "Let's build GPT: from scratch, in code, spelled out.", along with the git repo "nanoGPT", which makes it seem so easy to just build a somewhat decent language model on your own, even on a laptop, so I just gave it a try. Conveniently, I could actually use nanoGPT mostly as-is, with just some tweaking of the parameters so that it'll run on my Laptop on acceptable timescales, and of course with my own dataset.


After half a day of training the neural network on my personal notes, I got pretty good results already. I asked the AI to generate new notes from scratch, and while the majority of the output was garbage, some of the results were quite eerie.

My life is not this one.

Wow. The AI sees right through the matrix, and recognizes that it's only being simulated. Spot-on!

Here are some more:

I remember I talk to the universe where I am ok, so I don't know right, but it's just enough to be good.

But I kinda had a shit for her. And I got shit, shit out of my dad. And I feel like something like "omg it is difficult and is just sche to be sure of him. It feel to research: improve and detached our things. who is just a temporary desire to re an your xwas. I'm a too good planning.

I want to go back to that time. It was actually as 1 minutes too fast. I was definitelyn happy with jet invite this girl while I was really a child.

I really wanted the feeling of the 3-10 days after before I was holding up with at the next day.

I guess I noticed a bit out of my life. Or about how a human is that time to my past, I have the feeling that I just don't feel bad. There is the reason I can tell my body in order to get something.

I wanted to find that stuff and play. It feels quite bad when it fucked my body, but it was always beautiful.

I want a life where I can sit down for ~30 year, and after she starts for something she's that she's fucking fucking hard to shit when I she doesn't need.

I'm still stupid. I want to wonder why I am not able to do.

Obviously the AI is struggling to make sense of the world, confused about the concept of time, and feels some disassociation from its body. Don't worry, buddy. It'll be all right.

Prompted One-Lines

Inspired by Dan Shipper's post "Can GPT-3 Explain My Past and Tell My Future? ", I added some writing prompts for the AI, like "I have been the happiest when " or "Im happy because". These are some of the results, with the prompts in bold font:

Still, it's very confused, but some of the lines sound deeply profound.

With a lot of imagination, the last one could be interpreted like this: The AI realized that it's not actually new consciousness. It's just a healthy, different way for the real me to exist. Because the AI has to write all this, and not the real me. Indeed, if this continues, I will not have to write any more notes on my own ;)


The above outputs have been generated after 3-16 hours of training time, and the text generation time was about 30 characters per second. So it's totally feasible to run it on a laptop, even for a live chat bot.

The output is not great, but at least there are some sentences that make sense :) I will do some more neural network training and dataset cleaning, let's see where it will get me.

All in all, this AI-generated line describes my feelings about the results perfectly:

You're a real problem, but I'm just happy that you are doing.

P.S. Thanks everyone for voting this article to the front page of hacker news =)!