Liberal Democratic voters often express frustration that their Congressional leaders spend all of their time pandering to the right wing of the party. Conservative Republicans feel similarly about their folks worrying too much about appealing to the center- something that's playing itself out in NY-23 right now.
(Note that I personally think the Republican leadership is plenty friendly to the far right, but the far right doesn't, so my points in this post operate on the assumption that they're correct.)
What compounds the frustration for these groups is that they are indeed more numerous within their parties than the folks who are getting pandered to. 22% of Democrats think that the party's too conservative, only 15% think it's too liberal. 35% of Republicans think that the party's too liberal, only 20% think it's too conservative.
Despite their greater numbers though the truth at the end of the day is that Democratic leaders can take their liberal wing for granted, and Republicans can do the same with their conservatives.
Democrats who think the party's too conservative still say they'll vote Democratic by an 86-2 margin next year. Democrats who think the party's too liberal only say they'll vote Democratic 48-45. Who do you think leadership is going to worry about keeping happy?
Same story with the Republicans. The folks who think they're too liberal will still support the party by a 90-2 margin. The folks who think it's too conservative only give the GOP a 52-16 margin of support for next year. Again, which group is going to make party leaders more nervous?
In a two party system liberal Democrats and far right Republicans will continue to get taken for granted unless a) third party alternatives become more viable or b) serious, well funded primary challenges to incumbents become considerably more common. We are seeing that happen in NY-23, whether it becomes true on a broader scale remains to be seen.