To terminate a protected mode program execute an Int 21h with AH=4Ch in protected mode. You can return an error code in the AL register. This is the standard DOS terminate API but it must be executed in protected mode to allow the DPMI host to clean up any data structures associated with the protected mode program.
Programs should not be terminated from a hardware interrupt, exception handler, or real mode call-back. Programs should only be terminated from their main thread of execution to allow the DPMI host to clean up properly. However, DOS extenders that use the raw mode switch services for all mode transitions can execute the terminate call after switching from real to protected mode.