Документация говорит, что в принципе не нужно вызывать, но иногда нужно.
Обычно код выглядит

~Handle()
{
 if(handle != NULL && handle != INVALID_HANDLE_VALUE)
  CloseHandle(handle);
}

Получается, что проверка на INVALID_HANDLE_VALUE лишняя ?

Generally, an application should call CloseHandle once for each handle it opens. It is usually not necessary to call CloseHandle if a function that uses a handle fails with ERROR_INVALID_HANDLE, because this error usually indicates that the handle is already invalidated. However, some functions use ERROR_INVALID_HANDLE to indicate that the object itself is no longer valid. For example, a function that attempts to use a handle to a file on a network might fail with ERROR_INVALID_HANDLE if the network connection is severed, because the file object is no longer available. In this case, the application should close the handle.