I got it. Your phones talk over the internet to their PBX system via VoIP. With no quality of service across the internet, plus the fact that incoming phone calls to their PBX equipment are not prioritized based off of who they come from, I don't think tagging your packets will have much effect. VoIP is an RTP stream that goes in one direction at a time. It uses UDP to send a flow of packets that, if interrupted, could cause poor quality calls, or audio/video for that matter. It doesn't matter how much you compress it, it's still unidirectional and can be influenced by heavy traffic.
With that in mind, you will need to prioritize incoming and outoing voice traffic so that it gets queued and sent first instead of internet traffic. This will help if someone is downloading a big file across the internet. Prioritization will need to take place on the farthest equipment to the ISP. In other words, if you are running DSL and have a DSL modem, whatever device that sets directly behind it will need configured unless it is a multifunction device. In turn each piece of equipment coming back into your network will also be required to support QoS as well and be configured or this won't work. The endpoint QoS device will be the switch that the phone is plugged into. Since your phones have their own vlan and are basically on their own subnet, I would probably prioritize by IP source/destination. Do you know what IP address return calls will be coming from? If someone picked up a phone at home and called your office, what IP address would they show as coming from? This will help with creating your QoS on your network. Also, do your phones have live IP addresses on the internet or are you doing a one to one NAT for them?
