I would modify it without using the customer group or segment. Leave the segment for CRM and the customer group to control posting (unless there is a 1:1 relationship here). You need to do this on an item/customer level I presume. Nothing as standard would do this.