ProxyPass

[dlhhyct]

when exploring ajax and XmlHttpRequest(XHR), I realize that browser by default doesn't allow sending XHR to other domains, so one solution is to proxy the site I try to query. so in mysite.com rails app, I like map all request starting with "mysite.com/yahoo" to "api.search.yahoo.com", so XHR call to "/yahoo" transparently gets content from api.search.yahoo.com and update browser UI through js.

I tried rewriteRules with no luck:
# has no effect.
ProxyPass /yahoo http://api.search.yahoo.com/

# this just redirect browser to yahoo, not what i want.
RewriteRule /yahoo http://api.search.yahoo.com/

# caused 500 error, and log ask me to setup external webserver app
RewriteRule /yahoo http://api.search.yahoo.com/ [P]

what's the right way to do this?

[dlhhyct]

to make it simple, in my website, i want all my request
"mysite.com/foo" gets content from "other.website.com/bar", what's the best way to do it?

for example, i want my site users stay on my site,
"mysite.com/blog" point to "dlhhyct.blogsot.com",
so "mysite.com/blog/2006/10/rant" gets content from "dlhhyct.blogsot.com/2006/10/rant",

proxy external website only allow me to enter ip address, not website address. I want to be able to use website address.

[xing]

it's not possible to setup a proxy server by entering a domain name or an url.

LiteSpeed is not designed to be a client proxy which is basically what you are trying to do.

[dlhhyct]

http://httpd.apache.org/docs/1.3/mod/mod_proxy.html#proxypass

Apache can do this with mod_proxy, so why there's no equivalent in lsws?

[xing]

If more of our users use and request this feature, then we will put it on our roadmap.

This proxy to URL feature is not supported because this type of proxying is not often used and does yet fall under our development mandate of putting in features that at least 70% of our users will find useful.

[dlhhyct]

I can hack around this by doing it myself in ruby, but i guess it's much slower than web server doing it.

as mashup and ajax become more and more popular, instead of just put up some static flickr and google map widgets on own sites. I think down the road more and more user will demand more interactive functionality without leaving the site, that means eventually, demand of cross-domain ajax call will make proxyPass a feature hard to ignore.

I guess I can wait for now.

[xing]

Mashups and AJAX are in the general sense API for the backend to interact with other external servers or with web clients. I still don't see how lack of proxy by URL is part of the problem. If your program is ruby, you would query Google/Flickr/etc API and return content back to client.

If you can explain in detail how you use URL based proxying to enhance your mashup/ajax application, it would greatly help us see the value you have for this feature and how applicable it is to other developers as well.

[dlhhyct]

in an ajax call, ajax XmlHttpRequest(XHR) call within own domain ("^/") is allowed, but call to an external website is denied by default browser security setting. One way to work around this is on your own webserver, map "/external" to "external_website.com", so XHR call can get content that way.

without proxyPass, I can just use ruby's http-access lib to do a wget of remote content, but it's not the best way.

Here's a conversation when people try to use Google web toolkit (GWT) and facing this problem. http://groups.google.com/group/Google-Web-Toolkit/browse_thread/thread/9b2ea12d69696c26/27a2c4dc34e67802?lnk=gst&q=proxypass&rnum=1

[dlhhyct]

and another use is just simply proxying,
"mysite.com/blog" point to "dlhhyct.blogspot.com"

[dlhhyct]

as I surf more and more. I see there's more use of proxypass.


http://blog.guykawasaki.com/ points to --- http://guykawasaki.typepad.com/blog/ ,
http://jobs.slashdot.org/ points to --- slashdot.jobthread.com


such private labeling is getting pervasive. So site owner doesn't have to download and install software to offer additional functionality or revenue stream, like slashdot jobs. and user stays on the site longer.

The reason I use lsws so I don't have to spend days to figure out apache, nor do I want to setup as additional servers in the middle between my site and external site. please don't let me switch back.

both apache, lighttpd has it. please implement this.

[dlhhyct]

after you read this, if you find such feature important to you, please reply with "I want it too". thank you.
-------------------------------------------------------------------------------------------------------

What is proxyPass?
It's a feature that allows your website to act as a facade that can get content from other websites. For example, if you have a site called xyz.com, and you want to add forum functionality, and you know coolforums.com offers that, then you open an account on coolforums, and in your site, declare "proxypass forum.xyz.com xyz.coolforums.com", then you got instant forum functionality, doing so hide the supplying site, but this double the bandwidth cost. another way is just to point the DNS address of forum.xyz.com directly to coolforums.com, but revealing the supplying site.

I'm a site owner, why do I need it?
It's the simplest way to offer additional functionality on your website, fast. no software to write/install, no hareware to buy, basically you totally outsource it to the utility site which offer the functionality, be it forums, job board, blogs, etc. If you have a high traffic website, then you can offer job board and start earning in matter of minutes this way. Using proxypass allow you to offer more website features in less time, so user will stay on your site longer instead of hopping site to site.

I'm a web developer, why do I need it?
Due to default security setting, browser by default doesn't allow sending XmlHttpRequest(XHR) to other domains, most user don't know how to change that or want to change that. so one solution is to proxy the site I try to query. so in mysite.com rails app, I like to map all request starting with "mysite.com/yahoo" to "api.search.yahoo.com", so XHR call to "/yahoo" transparently gets content from api.search.yahoo.com and update browser UI through js.
for detail post, see http://litespeedtech.com/support/forum/showthread.php?p=3472

Current status
lsws does proxy web server already, but only resolves to IP and internal network, not url address. I've asked this before, but not deemed as priority by the development team.

As I surf more and more, I find Proxypass is empowering a whole new generation of websites, which offer internet services meant to be private labeled, or used as utility, whatever you call it. instead of creating the next blogspot.com, typepad allow user's blog to be integrated into user's own site. Instead of creating next monster.com, jobthread.com allow highprofile sites (slashdot, salon) to customize and integrate it onto their own site. Proxypass is doing all that magic behind the scene.


references
apache doc: http://httpd.apache.org/docs/1.3/mod/mod_proxy.html#proxypass

examples

http://blog.guykawasaki.com/ dns pointed to --- http://guykawasaki.typepad.com/blog/ ,
http://jobs.slashdot.org/ dns pointed to --- slashdot.jobthread.com

[mistwang]

The two examples you give does not implemented with proxypass, they are just domain alias for the same vhost, the two domain names point to the same server.

[rlance]

ProxyPass would indeed be helpful for Zope based sites using VirtualHostMonster.

Typically I run zope instances run on ports 8080 - 8100. The apache mapping of one such goes like:
ProxyPass / http://site.name:8082/VirtualHostBase/http/site.name:80/site/VirtualHostRoot/
ProxyPassReverse / http://site.name:8082/VirtualHostBase/http/site.name:80/site/VirtualHostRoot/

So, besides the current external Web Server 127.0.0.1:8082, it would be nice to allow the /VirtualHostBase/http/site.name:80/site/VirtualHostRoot/ as an additional part of the mapping of the site root.

[dlhhyct]

yes, you are right.

but if i really want to hide the supplying site, i can do it using proxypass. at the cost of doubling the bandwidth. I don't know if anyone does that now.

plus, it's a standard feature on apache and lighttpd, i don't understand why can't lsws have it.

[dlhhyct]

* http://blog.guykawasaki.com/ points to --- http://guykawasaki.typepad.com/blog/ ,

I thought about it again. Yes, i was wrong, i guess people dont do ProxyPass cross servers, since it's just not so efficient, unless they really want to. But I think they still do it locally.

here's how I will implement this: blog app is a virtual host taking wildcard domains, and looks for subdomain to serve the user. to accomodate outside domain names, add Proxypass that connect outside domain to matching subdomain, I think that's the simplest way.

Is there way to translate outside domain to matching subdomain without using Proxypass?
what's the right way to do it in lsws? is it possible to be done programmtically without manual input?

I'm asking for this because that's the direction I'm going, so this feature is really important to me. and I think that's direction the internet is going, more utility like.

[mistwang]

I think the example you given is just implemented with URL rewrite, no Proxypass involved at all.

[mistwang]

rlance,

you can use URL rewrite with Zope + FCGI for the same purpose.

[dlhhyct]

if u read the beginning of the thread, i tried rewrite without success. can you show me how?

what does proxypass do extra, that rewrite can't do? There must be a reason both apache and lighttpd have mod_proxy.

[dlhhyct]

as far as i know. url rewrite will redirect first link to second link, not proxying.

if anyone can use rewrite to do what proxypass can do, let me know.

[xing]

dlhhyct,

rlance's situation is that both vhost is on the same server. Why do you want to proxy when rewrite is more efficient? Your situation might be different as you are proxying out to 3rd party networks but when services on the same server, just different paths, proxypass is not the best answer.

[dlhhyct]

please read
http://www.litespeedtech.com/support/forum/showthread.php?p=3479

can you do what i try to do without using proxypass?
i tried rewriteRules, but can't get it to work.

[mistwang]

Rewrite only works when everything is on the same server, just the URLs/Hosts are different, as the example you gave.

As to what you want at the beginning of the thread, rewrite does not work. Maybe a reverse proxy to api.search.yahoo.com works, if the request without any change can be processed by yahoo.

For the forward proxy feature, it is not trivial, we can put it on our to-do list, but I am not sure when it will be implemented.

So, if the reverse proxy does not work, you need to setup Apache or lighttpd behind LSWS for the proxypass feature, or, you can also try some off shelf cgiproxy scripts written in perl or PHP.

Good luck. :)

[dlhhyct]

I have to find a good clean solution instead of just throwing whole bunch of stuff together. Thanks.

[rlance]

Yes, indeed, I would be grateful to learn how to accomplish the forward and reverse proxy via rewrite. Could you provide an example?

[SoreGUms]

Resurrecting an old thread is always fun... I need to do something similar to the OP.

In apache you do this:

ProxyPass /pytest/ http://192.168.1.8:8086/py/
ProxyPassReverse /pytest/ http://192.168.1.8:8086/py/

ProxyPass /pydev/ http://192.168.1.34:8086/py/
ProxyPassReverse /pydev/ http://192.168.1.34:8086/py/

This allows me to use the following URLs in my AJAX comms:

http://mysite.com/pytest/ - goes to the .8:8086/py/ host - in the log file the request is for /py/ not /pytest/
http://mysite.com/pydev/ - goes to the .34:8086/py/ host - in the log file the request is for /py/ not /pydev/

How do I do the above in LSWS?

As it stands in v3.3.15 I see I can add
TEST = 192.168.1.8:8086
DEV = 192.168.1.34:8086
as webservers. Then on the vhost mysite.com I can set a proxy context as
URI: /pytest/
Webser: TEST

but when I do
http://mysite.com/pytest/
the result that gets passed through is
http://192.168.1.8:8086/pytest/
I need that to be
http://192.168.1.8:8086/py/

How do I do what I want in LSWS? At the moment it looks like I don't and I uninstall LSWS and use Apache or something that can... I don't want to do this
Thanks :)

[mistwang]

With Enterprise edition, you should be able to rewrite the URL from "/pytest/" to "/py/" with rewrite rule, however, LSWS will not do any link rewrite in response body.

[Marcus]

NginX can too, and I agree that it would probably be a very useful feature.

It should be pointed out, though, that if it were to be implemented, that users should make sure not to turn their server into an open proxy unintentionally. Many hosting companies would have problems with this, and it would negatively impact on server performance. This might be something to mention in the documentation, were it to be implemented.

[mistwang]

Nginx proxy module is "an HTTP/1.0 proxy without the ability for keep-alive requests", it is a reverse proxy, not a squid like forward proxy.

The same function is present in LiteSpeed long time ago, and HTTP/1.1 keep-alive is always used for maximum performance, even when the client request is HTTP/1.0.