MSDN:

You can use the CGdiObject::DeleteObject function to delete a bitmap loaded by the LoadBitmap function, or the CBitmap destructor deletes the object for you.

Before you delete the object, make sure it is not selected into a device context.