|Subject:||Re: [Catalog-sig] Rewrite PyPI for App Engine?|
|Date:||Fri, 25 Jun 2010 11:49:16 -0500|
On Fri, Jun 25, 2010 at 3:39 AM, M.-A. Lemburg <mal@xxxxxxxxxx> wrote:
If GAE was just another hosting system, then sure -- but it's not.Â For instance, Noah mentioned if Apache went down (or the equivalent) there's someone with a pager who will respond to it.Â Except GAE isn't actually like that; application instances are can be automatically killed, machines are monitored automatically and brought out of the pool as necessary.Â We're not replacing our diligence with Google employees, it would be replaced with machines.
Of course there might be network problems or Google's own problems growing the service.Â But a substantial class of problems (problems that I believe have actually caused downtime) are simply eliminated from the system.Â GAE has less serviceable parts; that appears like losing control but it's really the normal progression away from manual interactions.Â I would really like if there was an open source alternative that provided that kind of infrastructure, but there isn't.
Another advantage to GAE is that if there are application errors, it would be much easier for anyone to work on them -- anyone can sign up and receive a free GAE account and deploy the code with almost no effort, and they will be hosting that is completely equivalent to anyone else's hosting.Â The only difference would be the data set, and it is possible (maybe even likely) that some class of problems will only be noticeable with a full dataset.Â That's true now as well, like for some UI problems where pages have become unwieldy, and I think it would be really helpful (regardless of GAE) if PyPI had a cleaned-up-export built into it.
Other cloud service providers provide something very different from GAE, and I don't think they would give a lot of benefit.Â The one advantage I see is that we (well, anyone) could spin up a new instance in a consistent state.Â Everything else is basically the same, including all the same management issues -- there's no one to kick Apache except us, for instance.Â Honestly if I have any skin in the game it's actually for a system like this, as I've been working on this sort of infrastructure (http://cloudsilverlining.org) -- I only propose GAE because I genuinely think it will work best for a volunteer-run piece of infrastructure like PyPI.
We have to find a middle ground, where we can still apply the
That's the catastrophic case, where Google decides they don't care about App Engine or something like that.Â Right now we'd have to do the same thing if the server's hard disk dies, which is obviously far more likely.
If we were to go for a cloud service to deploy the PyPI runtime, I'd
Setting up infrastructure for fail-overs is hard, and it would be easy for us to set it up for the wrong pieces (the ones that aren't breaking).Â In some sense this is why I'm not excited about mirroring, because I don't think it's fail-over for the pieces likely to break.
I do like the static file proposal, also.Â I think just putting more content into static files could potentially fix most of our problems, along with maybe a bit of server tweaking (to make sure even if PyPI goes down, it doesn't take Apache and the static files with it).Â I think using a CDN would be a nice step for speed, but is less important for reliability; I think generating things with a cron job will reduce reliability because it's exactly the kind of behind-the-scenes machinery that could break without someone noticing, and we don't have a dedicated staff paying attention to things like that.Â If a new package registration breaks, I'd far rather it be rejected immediately (e.g., from setup.py register) than for a broken cron job to keep it from getting in the simple index.
Ian Bicking Â| Âhttp://blog.ianbicking.org
_______________________________________________ Catalog-SIG mailing list Catalog-SIG@xxxxxxxxxx http://mail.python.org/mailman/listinfo/catalog-sig
|<Prev in Thread]||Current Thread||[Next in Thread>|
|Previous by Date:||Re: [Catalog-sig] Rewrite PyPI for App Engine?, Jesus Cea|
|Next by Date:||Re: [Catalog-sig] Rewrite PyPI for App Engine?, Terry Reedy|
|Previous by Thread:||Re: [Catalog-sig] Rewrite PyPI for App Engine?, M.-A. Lemburg|
|Next by Thread:||Re: [Catalog-sig] Rewrite PyPI for App Engine?, "Martin v. LÃwis"|
|Indexes:||[Date] [Thread] [Top] [All Lists]|