Seem like you have a SIP trunk issue. One side of your trunk is not sending a BYE message. Check with your SIP trunk provider for the proper asterisk/trixbox trunk setting. Or check on the website or google for "SIP trunk hangup problem"
BTW, you can terminate the conference by double clicking it in the trixbox panel. This way you will not need to reboot you trixbox.
Good luck.