Sounds like the little shutoff plunger or washer is loose. May be a bit worn out too.
Before you do anything else, I'd suggest you check and make sure the float is travelling freely righ up to the top. If there is an adjustment for how high the float must travel, turn the screw clockwise one turn or less. This will shut the water off a fraction earlier.
Answers & Comments
Sounds like the little shutoff plunger or washer is loose. May be a bit worn out too.
Before you do anything else, I'd suggest you check and make sure the float is travelling freely righ up to the top.
If there is an adjustment for how high the float must travel, turn the screw clockwise one turn or less. This will shut the water off a fraction earlier.