Friday, August 26, 2022

TimeSnapper: The power of being able to replay the past

To get this disclaimer out of the way right out front: I'm the author of TimeSnapper for MacOS. But like the guy in the 1980-something Hair Club for Men TV commercial: I'm not just the author, I'm also a client! A real event just happened where TimeSnapper enabled me to share a key bit of information with my team at work that I hadn't been able to come up with otherwise.

About a month ago, I came across a mention from a co-worker of a book as being a great introduction to the FinTech space: "The Anatomy of the Swipe" (2020) by Admed Siddiqui. At the time, I checked my local public library for a copy, but they didn't have one; so I took advantage of an awesome feature of the library, and put in a request via the library's website for the library to purchase a physical copy! They did so, and automatically put the book on hold for me! I picked up the book yesterday.

In a team meeting near the end of my work day yesterday, I mentioned the book, and promised to drop a link to the book in our Slack channel in the morning. I followed up on that this morning, intending to post the link along with the original post where I'd seen the book recommended internally.

I searched Slack for the earlier mention of the book that I remembered, and... nothing. 

No search results for "anatomy of the swipe". Or for "anatomy" "swipe". Or for "ahmed siddiqui".

Although I thought I'd seen the book recommendation via Slack, maybe it had actually been via shared Google Doc...? Or an email? I searched those places too, and... again, nothing.

At this point, I was momentarily stumped. I wanted to provide credit where it was due and share the original book recommendation I had followed... but where was it? Obviously I hadn't just imagined it; I had gotten the name and author of this particular book, in order to request my local library to order a copy, from somewhere.

And then, inspiration struck: I've got TimeSnapper! TimeSnapper, if you're not familiar with it, is a utility program that runs in the background on your computer, and saves a snapshot of the screen every few seconds. It also provides a "Play Your Day Like a Movie" feature, where you can rapidly scrub back and forth through an entire day's worth of these saved screenshots.

To get the time window for my TimeSnapper search, I went to my library's website, logged in, and and viewed the page with the status of my past purchase requests. Sure enough, it showed that my request for "The Anatomy of the Swipe" had been originally made on July 26, 2022 -- one month ago today.

I then fired up TimeSnapper's Play Your Day feature and navigated to July 26. Skimming through my day, I found the images of my visit to the library website where I'd initially keyed in my request for the book. Navigating a short way back in time from there, I saw myself visiting my company's Slack instance... and found the recommendation I had remembered -- not for the book itself, but a link to a blog article with a list of good FinTech introductory resources -- which is where the mention of the book has actually been located!

Thanks to TimeSnapper, I was able to post a link that Slack post with the link to the FinTech blog post along with the link to the book itself.

Obligatory closing promotional line: If this ability to go back in time and observe what you were seeing on your computer in the past sounds like a superpower you'd like to gain for yourself, you can read more about TimeSnapper, and buy a copy for Mac or for Windows, on the TimeSnapper website!

Friday, April 29, 2022

Nervous Flyer + Gamer? Try Playing Mario Kart

Hi! I'm Jon, and I'm a nervous flyer. 👋 Always have been!

Don't get me wrong: I totally understand and agree that riding on a commercial airliner is generally one of the safest activities that one can engage in. It's a lot safer than driving a car, for example.

Against my will, though, while flying -- especially during takeoffs, landings, and during turbulence -- some primitive part of my brain insistently alerts me that I'm in danger. Even though at the same time, I know that I'm not. It's pretty weird and annoying.

Past attempts to cope with this by distracting myself with movies, reading, or playing video games generally haven't helped.

In recent years, though, I've found a workaround that, quite oddly, actually is fairly helpful: Playing Mario Kart!

Luigi in a kart on the "Sunshine Airport" course of Mario Kart 8 Deluxe

For whatever reason, while I'm playing a Mario Kart race, that primitive part of my brain that normally sounds the alarm as sensations of motion are happening while in flight, instead seems to interpret those sensations as motions that are happening in the game -- and the unpleasant "danger" feelings are largely ameliorated!

I've found that this works for me both with the latest Mario Kart game, Mario Kart 8 for the Nintendo Switch, as well as with the older (and cheaper!) Mario Kart 7 for the Nintendo 3DS.

To maximize my time spent in races (instead of between races), as well as to enjoy the gameplay more, on my airplane ride Mario Kart 8 games, I've taken to using the "custom rules" game mode (found on the game's main menu under Single Player > VS Race):

  • Items: "Mushrooms Only." More opportunities to take those fun course shortcuts that require a speed-boosting mushroom. And I don't miss having my racing interrupted by lightning bolts and blue shells!
  • Race Count: Depending on the available time, I pick some high value, like 24 or 48.
  • Course Selection: "Automatic - In order." 

Would this trick work with other racing games, too? Good question! Maybe? Mario Kart is the only driver-perspective (or behind-driver-perspective) racing game that I've spent more than a short time with over the past few years.

If you're a fellow nervous flyer who is also a gamer, consider giving Mario Kart a try next time you find yourself up in the air!  (And don't forget to bring a USB battery pack to keep that Nintendo Switch charged!)

Saturday, April 23, 2022

Investigation and fix: A SQL Server stored procedure slowdown

What happened?

At work this past week, I was alerted to a problem where a background job which processes incoming messages from a particular queue had slowed down dramatically. As a result, a significant delay had developed in providing certain event notifications to our customers.  

Our monitoring tooling showed that the culprit was a particular SQL Server stored procedure -- one that gets invoked as a part of processing each message from the queue. Execution time of that sproc had suddenly increased from near-instant to an average of 4 seconds, with spikes up to around 20 seconds. (The queue processing job is multi-threaded, so we were still processing about one message per second, but that was no longer keeping up with the incoming volume of messages during the busy hours of each day.)

No code deployments or changes to the environment had been made around the time of the initial slowdown; this problem had started occurring without an obvious root cause. 

Our investigation

The stored procedure in question comprises about a dozen individual SQL statements and queries. It wasn’t immediately clear which individual statement in the sproc was responsible for the slowness.

After a few initial investigative paths didn’t yield results, I decided to run a short trace on the live production database. The trace revealed that one particular SELECT query was consistently taking a long time to run during each run of the stored procedure.

Interestingly: When I ran that same SELECT query SQL manually, it completed near-instantly. However, when I ran some minor variations on that SELECT query, they did take a long time to execute.

Execution plans and indexes

SQL Server internally generates an “execution plan” for each query that it runs, whereby it decides how it will go about performing the operation that the query's SQL calls for. The SQL Server Management Studio tool is able to display these execution plans in a nice graphical format.

When the problematic SELECT SQL was run manually, the execution plan showed that it was – correctly – leveraging an existing index on the target table. That index had been originally added to support this specific query. It includes fields needed by the query’s lookup criteria.

Looking back at the results of the trace, the problematics SELECT queries being run during execution of the stored procedure were being run using a different execution plan – not leveraging the custom-designed index at all! A full table scan was being run by SQL Server to gather the query results. As that  table currently sits at a total row count of over 150 million, those queries were taking a while to run!

The specific SQL being run (simplified a bit here for readability) was:

SELECT TOP 1 ID
FROM myTable t
WHERE t.CreationDate > DATEADD(day, -30, GetDate())
  AND (t.Email = @Email OR t.PhoneNumber = @PhoneNumber)
  AND t.CompanyID = @CompanyID)
ORDER BY t.ID ASC

The custom index includes the following fields:
  •     CompanyID
  •     Email
  •     CreationDate
  •     PhoneNumber

Can you spot the problem?



Take a look at the ID field. That field isn’t part of the index; but it is being used in the SQL’s ORDER BY clause! And that clause, as it turns out, is used by the database as one of the factors in how it seeks for the single (TOP 1) row to return.

Through its own inscrutable internal algorithms, prior to April, SQL Server had been using an execution plan that did leverage the custom index for this query, despite the presence of the ID field in the ORDER BY clause. After April, some tipping point in the database’s characteristics was evidently hit that made SQL Server decide to no longer use that index in its execution plan during runs of the stored procedure.

The fix was to adjust the SELECT query to ORDER BY CreationDate, rather than by ID. This had the same logical effect with regard to selection of the returned row; and it got SQL Server to once again use our custom index while running the stored procedure, resulting in the resumption of speedy performance in processing of messages from the queue.

How did we verify that the fix worked?

  • We observed that new records continued to be inserted correctly (and now, a lot more quickly!) into the database table.
  • Execution times for the stored procedure went way down. We could see this both from the database itself, and from the our queue's web interface – the green line here is the rate of messages being processed off the queue, with the fix having been applied just after the 10:20 mark:

So what are the lessons here?

  • Having the ability to determine which specific SQL statement was the root cause of the downstream slowness was key to being able to devise a fix. (The fact that our monitoring instrumentation only reported to the granularity of which stored procedure was running slowly -- but not which individual SQL statement within the sproc was slow -- made this trickier.)
  • The availability of multiple weeks of monitoring data from our instrumentation did make it quick and obvious to confirm that there had been a dramatic change in performance from one of our stored procedures, which was the cause of the problem.
  • When a particular SELECT statement is slow, ensuring that a properly-matching database index is in place – and that index is being applied to live queries in the expected manner – is a very good thing to check!

Monday, February 28, 2022

Tip: How to easily reorder bullet points in Google Docs

There's an easier way to reorder bullet points in Google Docs than cutting and pasting!

  1. Position the insertion point over the bullet point line you want to move. 
  2. Press Ctrl+Shift+⬆ (up arrow) to move the line up, or Ctrl+Shift+⬇ (down arrow) to move the line down.

I’ve found that this works a lot more nicely than cutting the item to be moved, and then pasting the item in the new location in the list (since often, line breaks need to be fixed up after the new item is pasted in).

If you select multiple items in the bulleted list first, then the aforementioned keyboard shortcuts will move them all together, as a group. 

This tip works for reordering or moving regular paragraphs, too, in addition to bullet points.

Want to do the same thing in Microsoft Word? Check out my (much!) earlier post, Tip: How to easily reorder bullet points in MS Word, from 2009!

Sunday, December 26, 2021

My Game of the Year Awards: 2021

Dating back to my days as a kid, one of my favorite hobbies is computer and console gaming. Following the Gamers With Jobs rules -- only games that I played for the first time in 2021 are eligible to make the list -- here are my top 10 favorite new-to-me games of 2021 (along with the platform on which I played each game), in descending order:

1. Metroid Dread (Nintendo Switch)

After nearly 20 years, we finally got a new 2D Metroid sequel. And it does so many things right! Gameplay that deeply respects prior games in the series, yet smartly blazes new territory. Beautiful and finely-detailed graphics. Good map/level design. Tough but fair boss battles. It's probably impossible for the greatness of Super Metroid (SNES) to ever be matched again, but Dread is an excellent modern take on the formula.

2. Bravely Default 2 (Nintendo Switch)

Bravely Default 2 takes the raw materials of JRPGs going all the way back to the first Final Fantasy games, iterates and folds them back on themselves many times, and ends up with this pretty nice piece of gaming origami. The most important thing I look for in a JRPG is combat that is fun, well-balanced, and (ideally) rewards smarts/creativity on the part of the player, and Bravely Default 2 delivers.

3. Final Fantasy Pixel Remaster (iPhone)

Speaking of the first Final Fantasy games: It was a treat this year having in my pocket this update to the very first Final Fantasy featuring a wonderful soundtrack, sharp graphics, and modernized and rebalanced gameplay -- while still remaining faithful to the original. I don't often blog about individual games, but I did do a compare-and-contrast between Final Fantasy Pixel Remaster and the NES original.

4. Monster Train (Windows)

With my primary computer these days being a Mac, I don't do a lot of gaming on Windows, but I made an exception for Monster Train. It's a fun and smart iteration on the Slay the Spire deck-builder-battler genre. (I was initially put off somewhat by the protagonists being the forces of Hell, but it's fine; in Monster Train, the angels are a bunch of jerks, and your team is fighting for self-preservation.)

5. Xenoblade Chronicles (Nintendo Switch)

I'm still midway through this expansive classic, and it might end up higher on a hypothetical 2021-2022 combined list, but for now, here we are! I had originally passed on this game, but the combination of my teenage son recommending it to me as having "single-player MMO" combat, plus wanting to learn more about Shulk from Smash Ultimate convinced me to give Xenoblade Chronicles a fair shake, and I'm glad I did. And I really like the soundtrack!

6. Etrian Odyssey 5 (Nintendo 3DS)

I dusted off my 3DS again during a period where the aforementioned teenage son was monopolizing the household Switch, and discovered that he'd purchased a digital copy of Etrian Odyssey 5 (2017) at some point. I'm only midway through, but the tough combat and old-school DIY mapping using the 3DS's stylus are just as engaging as I remember from earlier series entries.

7. New Pokemon Snap (Nintendo Switch)

I had low expectations for New Pokemon Snap, but they were wildly exceeded when I had a couple of weeks to give it a try on a check-out from my local library. I think the thing I like most is the fact that when you manage to get a great photograph, it's 100% due to your own timing and skill -- not due in part to some huge passive buff from a piece of equipped gear, or a lucky roll on some behind-the-scenes RNG. I'm planning to purchase a copy of New Pokemon Snap for myself when it goes on sale.

8. Steamworld Quest (iPhone)

This deck-builder-battler that's not a roguelike, but is instead a party-based RPG, played really well on the iPhone platform, even though it was originally designed for larger screens. The unique battle system kept me engaged all the way through to the end of the game.

9. Roguebook (Mac)

Another fun riff on the deck-builder-battler-roguelike genre, adding a map exploration mechanic that ends up working a bit like FTL, where you need to do all you can in a given area to power yourself up, before taking on the area boss and advancing. A few remaining rough edges in both minor UI bugs and gameplay balance issues don't hold Roguebook back from being fun to play.

10. Sorcery! 4-game series (Mac)

Also available on iPhone, this isn't just a remaster of the original 1980s gamebook series, but a full remake, taking advantage of the digital platform. A combat system that incorporates a bit of skill, and isn't simply dice-rolling -- and smartly, gives the player the option to immediately replay battles that didn't go so well -- adds to the fun.

Honorable mention

Best new DLC: Fighters Pass 2 for Super Smash Bros Ultimate (Switch). I find it crazy how well this mashup of 82 or so distinct playable characters from dozens of different series actually works in practice. My favorite gaming experience of 2021, eclipsing any of the new games mentioned above, is playing 2v2 local-vs-online matches with the teenage son. (The characters I mainly play are Samus and Ludwig von Koopa!)