Working out a custody and visitation schedule with your ex was hard enough -- the least they could do is abide by it.
Unfortunately, when one parent is interfering with custody or visitation, there are no easy answers to such a complex emotional and legal problem. But there are some better and worse things to do if your ex is denying visitation time.
Many parents' first reaction is to deny child support, if possible. As the thinking goes, if I stop paying, maybe the ex will relent and give me more visitation. But there might not be a worse idea. Issues of child support and child visitation, at least in the eyes of the court, are treated as completely separate legal issues. And if you fail to meet your child support obligations, it will only result in punishment from the court.
You may also think that if your ex doesn't have to abide by that visitation schedule or parenting agreement, then neither do you. This also couldn't be further from the truth. Trading violations of custody orders will only escalate an already volatile situation.
Talk to your ex. I know, this may not be the most appealing solution in light of the denial of visitation, but the more these conflicts can be handled between you two (as opposed to in front of a judge), the better. And if communication with your ex is proving too difficult or impossible, perhaps try and get a mediator, professional or not, involved. The key is to resolve the conflict as peacefully as possible, before emotions get out of hand and the legal system gets involved.
Document the times your ex is denying visitation, and make sure you keep accurate records of your attempts to visit your child. If you do need to file a complaint with the court regarding parenting time interference, you're going to want the evidence to back it up. (This is also where your good behavior in regards to the parenting agreement will come in handy.)
Child custody issues are fraught with emotion, which is why communication, de-escalation, and non-legal means of dispute resolution are so important. If that fails, you should contact an experienced child custody attorney to discuss your case.