If I understand correctly the .net runtime will always clean up after me. So if I create new objects and I stop referencing them in my code, the runtime will clean up those
The .NET garbage collector knows how to handle managed objects within the .NET runtime. But the Dispose pattern (IDisposable) is used primarily for un-managed objects that an application is using.
In other words, the .NET runtime doesn't necessarily know how to deal with every type of device or handle out there (closing network connections, file handles, graphics devices, etc), so using IDisposable provides a way to say "let me implement some cleanup of my own" in a type. Seeing that implementation, the garbage collector can call Dispose() and ensure that things outside of the managed heap are cleaned up.