A survey of residents has found less than 5% support a Local Plan bid for a housing estate on disused sports fields.