Russian Roulette in Testing

Reading Time: 4 minutes

Description

Russian Roulette, the dangerous Russian game that we are all familiar with from usually movies. The Urban Dictionary gives such small description; “A game where a revolver with one bullet is placed and spun. Then you take turns putting the gun to your head and pull the trigger. If it’s empty, you pass it on until someone dies or becomes extremely injured.

By now we have filled our heads up with this literature, let’s narrow the subject down on how I relate this blog post to the Russian Roulette. Well, the good news is that I’ll not suggest to play it of course, but it is a good metaphor of a situation that I have experienced and the word came to my mind. Here follows the background; recently I was reviewing my Colleagues’ PRs and I noticed such code snippets in the IT test that they were getting objects from positional access in list data structures and assuming that the objects in the zero, first and second positions allegedly had expected values.

After I read those test cases a few times, the notion of the Russian Roulette popped up in my head. Eventually I came up with proper solutions towards the prevention of such messy assertions. I’ll describe two situations along with my suggested solutions. Please assume that demonstrations simple and non complex to outline more the cause.

Last of all, to demonstrate my samples, I am using the mirror servers of Fedora Linux Distribution @ https://mirrors.fedoraproject.org/mirrorlist?repo=epel-7&arch=x86_64

Situation: Asserting a Single value in a Collection

While coding a method that gathers or filters a list from the persistence unit, we tend to iterate through the list, grab and assume that the Nth element will be our expected value. In essence, it may lead to a very naive behavior, here is a code snippet

@Test
    public void shouldFindPolishHostFromSecureEuropeanHostsRussianRoulette() {
        //given
        final String expectedAddress = "https://ftp.icm.edu.pl/pub/Linux/fedora/linux/epel/7/x86_64/";

        //when
        final List<Host> hosts = repository.findByAddressContaining("http");

        //then
        assertThat(hosts).isNotNull();
        assertThat(hosts.get(2).getAddress()).isEqualTo(expectedAddress);
    }

As you observe the code and apprehend the logic quickly, we are getting a list of servers that are non-secure only http protocol. Here we have a catch which is, we are very very sure that the second element in the list will be the expected one. In fact it can be, however here can possible things may occur in the future:

  • If you update the memory database package or switch to another one, the new alternative may return the values in a different order because of its internal design. Thus, your test will obviously fail,
  • When the next developer adds a new entry before or after your test data, such tests will most likely fail.

Solution: Implement ways to look for the Exact Figure

The title of the solution reveals the whole deal right. Which one is more feasible way to ensure integrity, assumption or precision? Obviously precision is the keyword here. Here is the code snippet:

 @Test
    public void shouldFindPolishHostFromSecureEuropeanHostsNonRussianRoulette() {
        //given
        final String expectedAddress = "https://ftp.icm.edu.pl/pub/Linux/fedora/linux/epel/7/x86_64/";

        //when
        final Host host = repository.findByAddress(expectedAddress);

        //then
        assertThat(host).isNotNull();
        assertThat(host.getAddress()).isEqualTo(expectedAddress);
    }

My suggestion is when we are exactly looking for a single value, we must be in true certainty, and look for an exact figure like in the above example. Implement methods that will query for certain values or search the persistence unit.

Situation: Asserting a Collection of values with hard coded values

Throughout my software development career, I always came across with such situations in which I have a list of expected values that must be ensured the computer had produced same or similar output. This situation can come from different varieties and sorts. Instead of discussing possibilities at the table, let’s look at this code snippet and begin a rare case:

 @Test
    public void shouldFindAllSecureEuropeanHostsRussianRoulette() {
        //given-when
        final List<Host> hosts = repository.findByAddressContaining("https");

        //then
        assertThat(hosts).isNotNull();
        assertThat(hosts.get(0).getAddress()).isEqualTo("https://mirror.karneval.cz/pub/linux/fedora/epel/7/x86_64/");
        assertThat(hosts.get(1).getAddress()).isEqualTo("https://ftp.icm.edu.pl/pub/Linux/fedora/linux/epel/7/x86_64/");
        assertThat(hosts.get(2).getAddress()).isEqualTo("https://ftp-stud.hs-esslingen.de/pub/epel/7/x86_64/");
    }

In this example, we are hard core assuming that the given Nth element will have such outcome as we type in. In this case I have the same failure point of predictions as I declared in the above situation, nothing more to address here.

Solution: Compare Collections with Collections

As we get the spoiler from the title wouldn’t it be nice to purely compare collections with collections? I must say, I really admire the way that I approached this situation and the solution as well as the testing frameworks support into it. Let’s apprehend my solution here

@Test
    public void shouldFindAllSecureEuropeanHostsNonRussianRoulette() {
        //given
        ArrayList<String> expectedHosts = new ArrayList<>(Arrays.asList("https://mirror.karneval.cz/pub/linux/fedora/epel/7/x86_64/",
                "https://ftp.icm.edu.pl/pub/Linux/fedora/linux/epel/7/x86_64/",
                "https://ftp-stud.hs-esslingen.de/pub/epel/7/x86_64/"));
        //when
        final List<Host> hosts = repository.findByAddressContaining("https");

        //then
        final List<String> hostNames = hosts.stream().map(Host::getAddress).collect(Collectors.toList());
        assertThat(hosts).isNotNull();
        assertThat(hostNames).isNotNull();
        assertThat(hostNames).containsAnyElementsOf(expectedHosts);
    }

The code is very simple to understand its purpose, in this solution I really don’t need to know in which position I get what outcome. The importance here is to ensure the computer had prepared the output that has the desired output, it as simple as that. Furthermore, the testing framework provides such method that ensures the output meets the expectation

Bonus Solution: Extracted Properties of Collections

I asked my colleague at work for a favor of evaluation my work and I was really curious about his opinion and thoughts. He had pointed me some features from the AssertJ library that I find very useful and the feature Assertions on Iterables can be quite comprehensive as well. In basic terms I also added a new test to simply cover the concept up

@Test
    public void shouldFindAllSecureEuropeanHostsNonRussianRouletteNonStream() {
        //given
        ArrayList<String> expectedHosts = new ArrayList<>(Arrays.asList("https://mirror.karneval.cz/pub/linux/fedora/epel/7/x86_64/",
                "https://ftp.icm.edu.pl/pub/Linux/fedora/linux/epel/7/x86_64/",
                "https://ftp-stud.hs-esslingen.de/pub/epel/7/x86_64/"));
        //when
        final List<Host> hosts = repository.findByAddressContaining("https");

        //then
        assertThat(hosts).isNotNull();
        assertThat(hosts).extracting("address")
                .containsAnyElementsOf(expectedHosts);
    }

In my sample I only demonstrated the “extracting” that eliminated my stream operation. On the other hand I’d like to share some links that will demonstrate way more advanced test cases than my humble test.

Conclusion

Assumptions can be evil in programming. Unless you work on sorted data structures that will guarantee the order, still assuming the positional access can be malicious, my sincere suggestion is that we shall always strive for best practices to overcome such bad habits, thus we won’t waste time on fixing unnecessary test cases. You can find the full solution at my Github repository @ https://github.com/tugrulaslan/BlogCodeSnippets/tree/master/RussianRoulette

Featured Image Courtesy telegraph.co.uk – https://i.telegraph.co.uk/multimedia/archive/02500/Russian_roulette_2500016k.jpg

Leave a Reply

Your email address will not be published. Required fields are marked *