The stage assigned as the current project stage cannot be made inactive. You would need to move to another stage first either by selecting any of the existing stages as the current project stage, or by progressing to the next stage.
Within the required Project click on the Stage Name to open the Stages Modal
Untick the [Active] flag on any of the stage you wish to make inactive and not currently assigned as the 'Current Project Stage' and [Save]
[Save]
Click 'Include Inactive' to display inactive Stages on the Project. These can then be made active again if required.