I am using the Zend framework with a front controller pattern. I am running the debugger locally in PHPED. When I start at index.php, I am ok debugging through the front controller and the main page. But on each link I have clean URLs so the debugger does looking for the full URL instead of passing the request back to my front controller (index.php) so every link shows a page not found error.

How do I configure the debugger to know that all URLs should go through the front controller?

Debugger is not supposed to be looking into that. What looks is browser and what responds is web server. Debugger works only after the script is run. Since your got "page not found", it's a clear indication that script was not run, so the debugger had no chance to help you.
I'd recommend you to learn how the wrong links were created and fix the problem.

I am running the Embedded Web Srv and kicking it off from the debugger icon (but of course the same failure occurs from running the app). As you point out the debugger itself is never invoked.

The tools..properties for the Web SRV has very little to configure (port, home dir, etc) so I assume that things like mod_rewrite are coming from an http.conf but I can't find one, other than that for my Apache installation. The only page that displays is the front controller because it is the only page that does not require mod_rewrite.

There are other configuration parameters required of the web server to work with Zend Framework. So my question still remains, can anyone tell me how to configure PhpED to work with Zend Framework?

First you need to run ZF. In this order please read ZF manual and at least install Apache if you need mod_rewrite as this module exists only for Apache. Then please read this FAQ on how to configure PhpED to work with Apache or IIS: http://forum.nusphere.com/howto-setup-phped-with-local-apache-or-iis-t647.html

Is this possible? Has anyone ever gotten it to work? The comments from dmitri suggest that he does not know it it will work.

It's possible and works fine.
http://www.nusphere.com/php/zend_framework.htm