Tutorials on Test

The proliferation of Large-Language Models (LLMs), and their subsequent embedding into workflows in every industry imaginable, has upended much of the conventional wisdom around quality assurance and software testing. QA Engineers effectively have to deal with non-deterministic outputs - so traditional automated testing that involves assertions on the output are partially out. Moreover, the input set for LLM-based services has equally ballooned, with the potential input set being the entirety of human language in the worst case, and a very flexible subset for more specialised LLMs. This is a vast test surface with many potential points of failure, one in which it is practically impossible to achieve 100% test coverage, and the edge cases are equally vast and difficult to enumerate - it’s unsurprising that we’ve seen bugs even in top tier customer-facing LLMs even amongst the biggest companies. Like Google’s AI recommending users eat one small rock a day after indexing an Onion article or Grok accusing NBA star Klay Thompson of vandalism .

If you work within a disorganized workspace with deeply nested folders and try locating a specific folder, file or code snippet, then your productivity suffers from the constant distraction of manually searching through the workspace. Navigating the workspace and rummaging through every folder (double-clicking each one) to find a single folder or file becomes repetitive and directs attention away from your work. If you forget to close the folders after exploring them, then these opened folders accumulate over time and obstruct subsequent searches by cluttering the screen. Additionally, a computer's file explorer, such as Mac's Finder or Ubuntu's Nautilus, slows down when loading and displaying folders and files within large external hard-drives, thumb drives or SD cards filled (or nearly filled) to maximum capacity. Operating systems based on the UNIX kernel provide the find and grep command-line utilities to search for files/folders and text within a file respectively via pattern matching. With a single-line command, you avoid interacting with the interface of the computer's file explorer. Instead, the command prints the search results to standard output ( stdout ) displayed within the terminal. Both the find and grep commands are considered as some of the most essential building blocks in bash scripting! Knowing how to use them allows you to integrate them into your continuous integration (CI) pipeline to automate search tasks. Below, I'm going to show you: