Optimizing C# Code: Using ‘yield return’ to Boost Performance

One of the trickiest C# concepts is yield return. For me personally, it has been one of the hardest concepts to grasp and now that I think I have understood it well enough to explain it, I will try to make it as clear as possible to anyone reading this blog post.

Per definition, the yield return statement in C# is used to create an iterator, which is a special type of method that can be used to iterate over a collection of items. When a method uses the yield return statement, it is called an iterator method.

An iterator method allows you to iterate over a collection of items without having to load the entire collection into memory at once. Instead, the iterator method returns one item at a time as you iterate through the collection. This can be very useful for large collections, as it can significantly improve performance by reducing the amount of memory used.

Lists

Let’s say you want to display the first 5 numbers to the users. For that you write the code below:

public class Program
    {
        static IEnumerable<int> GetNumbers()
        {
            List<int> items = new List<int>();

            for (int i = 0; i < 5; i++)
            {
                Console.WriteLine($"(GetNumbers) - {i}");
                items.Add(i);
            }

            return items;
        }

        static void Main(string[] args)
        {
            foreach (var i in GetNumbers())
            {
                Console.WriteLine($"(Main) - {i}");
            }
        }
    }

and if you run this code, the result would look like the below:

So, you can see that all the items were first added to the list, then all the items were read from the list. Now, let’s say you want to do the same for 2 million numbers. It means you need to operate over a collection of 2 million numbers. You’ll create all these items add them to the list, then perform operations on each of them.

This approach has two main disadvantages: it’s slow (imagine instead of numbers, you are working with a list of objects), and occupies a lot of memory.

This is when yield comes into help.

Lists with Yield

I will convert the code above to use yield. After the update it would look like below:

public class Program
    {
        static IEnumerable<int> GetNumbers()
        {
            for (int i = 0; i < 5; i++)
            {
                Console.WriteLine($"(GetNumbers) - {i}");
                yield return i;
            }
        }

        static void Main(string[] args)
        {
            foreach (var i in GetNumbers())
            {
                Console.WriteLine($"(Main) - {i}");
            }
        }
    }

Now if I execute this code, you will see a different result:

You can see that, with yield instead of creating the whole list, one item is created and returned at a time.

Some Real World Examples

Let’s say you are tasked to add logging and show the logs to your company’s flagship app. You log a lot of data, so you need to think of a solution to show logs to the users in a highly performant way.
In this case, you could use “yield” to return log files to the admin users. With yield return rather than loading the entire set of log entries into memory at once and showing them, you could use “yield” to return log entries one at a time.

Example code:

using System.Collections.Generic;

public class LogFileReader
{
    public static IEnumerable<string> GetLogEntries(string filePath)
    {
        using (var file = new StreamReader(filePath))
        {
            string line;
            while ((line = file.ReadLine()) != null)
            {
                yield return line;
            }
        }
    }
}

In this example, the GetLogEntries method reads a log file one line at a time using a StreamReader and yield return each line as it is read. This way, the entire log file doesn’t have to be loaded into memory at once, which can save memory.

Another example is in an application that processes a large amount of data, you could use “yield” to return processed items one at a time, rather than storing all of them in memory at once.

public static IEnumerable<ProcessedRecord> GetProcessedRecords()
{
    using (var connection = new SqlConnection(connectionString))
    {
        connection.Open();

        using (var command = new SqlCommand("SELECT * FROM DatabaseRecords", connection))
        {
            using (var reader = command.ExecuteReader())
            {
                while (reader.Read())
                {
                    yield return ProcessRecord(reader);
                }
            }
        }
    }
}

private static ProcessedRecord ProcessRecord(SqlDataReader reader)
{
    // perform processing on the record
    //...

    return new ProcessedRecord(..data...);
}

In this example, the ProcessData method processes each item in the input data one at a time using the ProcessItem method and yield returns the processed item. This way, the entire data set doesn’t have to be loaded into memory at once, which can save memory and time.