Restriction on the initial state for the absence of IR effect in single field inflation