On AI assistants and Kaprekar's constant
I was reading Programming vs. mathematical curiosity, where in the first few paragraphs we find out about Kaprekar's constant. Essentially, if you pick two four-digit numbers (including leading zeros) that have more than 1 unique digit, and "arrange its digits in descending and ascending order to get two different numbers... then subtract those two", you will eventually end up at the number 6174. It's an interesting thing to read up on.
Anyways, I read this and immediately had to implement it. As in, I didn't finish reading the blog post until I could prove to myself that this was correct. My solution is basically a brute-force solution with zero optimizations, which doesn't matter because it still runs as fast as it takes to build. Namely, I run through every valid number between 0 and 10000.
My code did, in fact, validate the theorem.
Of course, reading the rest of post afterwards, I found out the author did the exact same thing.
On AI assistants
I recently upgraded to a stat-maxed M3 MBP so I could play around with some LLMs locally. The first time I really played with any of the generative AI systems was in March [1]
I've been having fun figuring out how to make it useful. One example came up today: one of my pet peeves with any of the "smart" watches is that the battery life is pathetic. Well, except for my Garmin Instinct Solar: weeks of battery life, (very slow but still) solar charging, and does the few things I mostly want my wrist-computer to do. Namely, heart rate, sleep tracking, and tracking workouts.
Anyways, I was cleaning my apartment and found my Apple series 7 Watch. I started wearing it again, but it quickly turned into "charge the watch twice a day" and I quickly ended up trading it in for an Ultra 2.
All this to say that I wanted to figure out how long of battery life I could get out of it. Charging at most once a day is the ideal here, or in the case of the Garmin, damn near once a month. I looked up when my battery was last charged to 100%, and it was Monday at 18:31. Well, today (Thursday) at 10:31, I was at 35% battery and decided to charge before going to the gym. I posed the same question to Wolfram Alpha and llama3:70b, and they both got the answer wrong for different reasons. This difference is illuminating.
The prompt, by the way, is fairly naïve and innocuous:
How much time passed between 18:31 on Monday and 10:31 on Thursday?
So where did this go wrong?
On Wolfram Alpha's wrong answer
The problem is that this prompt results in Wolfram Alpha parsing "Monday" as "this Monday" as in Monday the 29th. The calculation here is this from 2024-04-25 10:31 to 2024-04-29 18:31. What I wanted was the difference between 2024-04-22 to 2024-04-25.
The error here is that Wolfram Alpha misinterpreted the inputs.
On the LLM answer
This one is a trip:
[2024-04-25 10:31] user How much time passed between 18:31 on Monday and 10:31 on Thursday?
[2024-04-25 10:31] assistant Let's break it down:
Monday, 18:31 to Tuesday, 00:00 (midnight) = 5 hours 29 minutes Tuesday, 00:00 to Wednesday, 00:00 = 24 hours Wednesday, 00:00 to Thursday, 10:31 = 10 hours 31 minutes
Adding up the time periods: 5 hours 29 minutes + 24 hours + 10 hours 31 minutes = 40 hours 0 minutes
So, 40 hours passed between 18:31 on Monday and 10:31 on Thursday.
[2024-04-25 10:33] user Wednesday to Thursday isn't 10h 31m
[2024-04-25 10:33] assistant I apologize for the mistake.
Let's correct it:
Monday, 18:31 to Tuesday, 00:00 (midnight) = 5 hours 29 minutes Tuesday, 00:00 to Wednesday, 00:00 = 24 hours Wednesday, 00:00 to Thursday, 10:31 = 28 hours 31 minutes
Adding up the time periods: 5 hours 29 minutes + 24 hours + 28 hours 31 minutes
Converting everything to hours and minutes: = 5 hours 29 minutes + 24 hours + 28 hours 31 minutes = 58 hours 0 minutes
So, 58 hours passed between 18:31 on Monday and 10:31 on Thursday.
[2024-04-25 10:36] user Take a deep breath. You're still off between Wednesday and Thursday, but we can do this. Instead of computing Wednesday 00:00 to Thursday 10:31, try Wednesday 00:00 to Thursday 00:00, then Thursday 00:00 to Thursday 10:31.
[2024-04-25 10:36] assistant Thank you for the clarification.
Let's recalculate:
Monday, 18:31 to Tuesday, 00:00 (midnight) = 5 hours 29 minutes Tuesday, 00:00 to Wednesday, 00:00 = 24 hours Wednesday, 00:00 to Thursday, 00:00 = 24 hours Thursday, 00:00 to Thursday, 10:31 = 10 hours 31 minutes
Adding up the time periods:
5 hours 29 minutes + 24 hours + 24 hours + 10 hours 31 minutes
Converting everything to hours and minutes:
= 5 hours 29 minutes + 48 hours + 10 hours 31 minutes = 63 hours 60 minutes = 64 hours 0 minutes
So, 64 hours passed between 18:31 on Monday and 10:31 on Thursday.
Thank you for your patience!
The error here is that the LLM failed at basic math.
Conclusion
I think it makes sense to use an LLM to design or enhance the prompt that is sent to Wolfram Alpha.
With this in mind, I'm thinking of building a personal assistant, and in the process of writing out design docs for it. In fact, I've spent a few hours using the LLM to explore the problem space, giving me notes and pointers to move forward.
| [1] | Of this year, that is. I've only really been playing with them for about two months. I get that I'm behind the curve. I did do a wee bit o' stable diffusion, but I didn't like the image resolution, and I never got around to running it locally. |