Thursday, August 13, 2020

Incrementally troubleshooting production server issues as sculpting marble

Near the beginning of Steve McConnell's classic book on software construction, Code Complete -- it's the one where Jeff Atwood's "Coding Horror" icon image originally came from -- several analogies for the process of creating software are considered, including penmanship, farming, and oyster pearl harvesting, before McConnell finally settles on "construction."

At my job, although I'm primarily a software developer, I do also occasionally put on the hats of DevOps specialist and DBA. In particular, this happens when a legacy server that I'm responsible for experiences production issues, and I'm one of the few with the knowledge and experience to figure out what's wrong, and get it up and running smoothly again.

This morning, while making another pass at troubleshooting a stubborn partial outage that had recently started cropping up on a near-nightly basis, where performance of the production website and/or database was being degraded in the early morning hours for a period of around an hour or two, another analogy for the type of production software maintenance occurred to me: that of sculpting!

Michelangelo's "David"
In troubleshooting this outage, my colleague and I have taken the approach over the past several days of of making a single change that might hopefully resolve the issue; then, waiting to see if the issue still recurred. If so, we'd make another change.
It occurred to me that this was not so dissimilar from the quote attributed to the sculptor Michelangelo saying, in reference to beginning a sculpture, that the desired figure is already in place; the excess marble that does not comprise the statue just needs to be chiseled away.
By way of example, here are some of the incremental changes we attempted, in working towards resolving our persistent nightly outage: 
  • Monitoring the performance of the database during the outage, we observed that a particular stored procedure was comprising much of the runtime.  We optimized that stored procedure by improving a couple of the existing table column indexes used in the sproc's query plan, to reduce the overall number of lookups the stored procedure needed to perform.
  • We altered one of our nightly scheduled jobs that was running concurrent with the outages to avoid calling the possibly-problematic stored procedure at all; instead, we fulfilled the job's data needs by adding some additional output fields to another query the job was already making.
  • Examining the raw server logs of incoming HTTP requests, we observed that a number of poorly-behaved bots (no user-agent strings; no reasonable rate limiting) were hitting our site very aggressively during the outage period -- and that the pages being hit were triggering calls to that same stored procedure. As at least an interim fix, we put rules in place to prevent the pages being hit from being rendered in response to those particular requests.

Each of these steps of trimming off poorly-performing code and problematic incoming requests has felt a bit like chiseling away some stone, hopefully in the end arriving at a nice statue -- or in our case, a stable production system!

Sunday, May 17, 2020

Review: Eufy Security Video Doorbell

This past November, I jumped on the trend of replacing my house's existing traditional doorbell with an Internet-enabled doorbell. The key reason for making the change was to solve the fact that I couldn't hear my home's doorbell chime at all while working in my basement home office, which was causing me to miss service appointments, deliveries, and visitors.

After some research, I decided to go with the Eufy Security Video Doorbell. I purchased it from Amazon on a Black Friday sale for US $108, discounted from the full sticker price of $160. (At the time of this post, Amazon is selling this doorbell for $136.)

The doorbell was easy to install. I'm only a very amateur electrician, but I was able to get it installed in place of my existing traditional doorbell with no problem. The doorbell comes with a high-quality set of instructions, in the form of both a short video served through the doorbell's companion smartphone app, and written instructions.

The doorbell's companion app works great for viewing recordings -- which are captured automatically when the doorbell detects a nearby person -- as well as for viewing a live video feed from the doorbell's camera.

The doorbell comes with a remote chime that plugs into a wall socket and plays a sound when someone rings the doorbell. A ring of the doorbell also generates a push notification from the smartphone app.

Both my wife and I were able to create individual Eufy accounts and then associate those accounts with our new doorbell, such that we were both able to receive doorbell ring notifications and view video from the doorbell from our respective smartphones, without having to share a single set of credentials, which was a nice feature.

A major benefit of the Eufy doorbell is that there's no ongoing monthly cost to get its full functionality -- unlike competing devices, such as Amazon's market-leading Ring doorbell. All of the video storage is onboard the device itself, not in the cloud. You still view video recordings and the live video feed from the Eufy app your phone, but there's no cost associated with that capability.

Another major perk of the Eufy device relative to the Ring is the relative lack of privacy / surveillance concerns. If that's something that potentially concerns you, a Google search on terms like "ring police surveillance" is a good starting point for doing your own research.

The one major caveat as of the time of this writing is that the doorbell unit stopped working a few weeks ago. It went from working one day to being dead/unresponsive the next -- no visible lights on the doorbell unit, and no response to button presses. I turned off my power and checked the wiring connections on the unit, but the issue still persisted after I verified that the wiring was fine.

When I contacted Eufy Support about this, they promptly sent me out a new unit, free of charge, which worked fine when I hooked it up to replace the existing (dead) unit. Given this A+ level of support, I don't consider this problem a significant con -- provided this replacement unit manages to last for multiple years!

As of this writing, I highly recommend the Eufy Security Video Doorbell if you're in the market for replacing your home's traditional doorbell with a video doorbell.

Thursday, April 23, 2020 now supports skin tones now supports your choice of skin tone! Just select a tone from the dropdown in the page footer, and the selected tone will be applied to all supported emoji.

In the course of developing this update, I was admittedly surprised to realize that the basic "facial expression" emoji -- πŸ˜„, πŸ˜‚, πŸ€”, and so forth -- aren't toneable! The emoji for which tone support has been added by the Unicode Consortium -- the ones in this list -- are mostly the "hand" emoji (e.g. πŸ‘‹πŸΎ, πŸ‘πŸΌ, πŸ™ŒπŸΏ), plus the ones that depict people's activities or appearances (e.g. πŸ‘©πŸ½‍🦳, πŸ‘¨πŸ»‍🏫, πŸ„πŸ½‍♀️).

The only emoji not supported by ClipEmoji yet are the three "holding hands" emoji (πŸ‘«, πŸ‘­, πŸ‘¬), where the two people depicted have different skin tones. The main thing to be figured out there is an elegant UI to apply tones to that emoji, that doesn't "get in the way" of the other thousands of emoji!

The ClipEmoji site will remember your selection for future visits to the site, too (via a cookie).

I also added an About page to, so check that out if you're in the mood for even more ClipEmoji-related reading.

Enjoy the updated! πŸ‘πŸ»πŸ‘πŸΌπŸ‘πŸ½πŸ‘πŸΎπŸ‘πŸΏ

Sunday, April 05, 2020 updated with 2020 v13 Unicode emoji set

I've updated with the new 2020 v13 Unicode emoji set. This is a new set of emoji, such as "smiling face with tear" (πŸ₯²), that will be added to various platforms -- iOS / MacOS, Android phones, Windows, and so forth -- later this year.

(So if you're reading this post just after it was published, you most likely cannot see that above emoji character quite yet! πŸ˜‚ ClipEmoji is now ready to go, though, as soon as support does get added to your platform.) is a simple and lightweight web page that's designed in particular for desktop and laptop computers -- where the keyboard (obviously) doesn't include emoji characters -- to make it easy to get a particular emoji that you want to use in some message or text you're composing onto your clipboard, so that you can paste it into the text you're composing.

Thursday, March 26, 2020

Side-by-side Firefox browsers separating work and personal accounts

Firefox is my favorite web browser. It supports proper Ctrl+Tab behavior, it's fast, and it's a leader when it comes to prioritizing the privacy of individual users over the agendas of corporate ad networks.

I also like keeping my work-related stuff and my personal stuff organized into two separate browser programs. A big advantage of that is I can be logged in to my work Google account (for email, calendar, etc.) in the first browser, and my personal Google account in the second browser, and don't ever need to manually switch back and forth between which account is currently active.

(Separate browser programs -- not just browser windows -- is needed for this to work. Separate browser programs, such as Firefox and Chrome, don't share cookie storage with one another, which means that they have completely separate records of your login information for each website.)

As a bonus requirement, I want to be able to tell at a glance whether a given browser window that I've got open in front of me belongs to my "work" or "personal" browser.

Finally, with Firefox being my preferred browser, I'd like to be able to use it for both my work stuff, and my personal stuff. Given that I need to be using two different browser programs in order to keep my work and personal stuff separated, is it impossible?

Enter: Firefox Developer Edition

As you've likely guessed, this all is possible, by installing Firefox Developer Edition!

Firefox Developer Edition is like the standard Firefox browser, but with a few additional programmer-oriented bells and whistles.  For most practical purposes, though, it works just like the standard Firefox.

Most importantly for this post's use case, Firefox Developer Edition fully supports being installed, and running, side-by-side with Firefox.  The two browsers each have their own separate cookie storage, and so each supports being logged in with a different user account to the same website, such as Google, at the same time.

Firefox and Developer Edition can also be themed (given a custom appearance) separately. I've landed on assigning a simple orange theme to my Firefox, and a simple blue theme to my Firefox developer addition, so telling which browser a given window belongs to is as simple as glancing at the top of the window, and looking at the color.

I'd encourage even non-developers who like Firefox, and are interested in separating out your personal and work stuff into separate browsers, to give this a try. There's nothing "scary" about Firefox Developer Edition; it's easy to just use it as another copy of Firefox.