Active Reload: Lighthouse Problems... Solved!

January 10th, 2008 by rick

Lighthouse Problems... Solved!

2 comments on 234 words

I just wanted to let everyone know that I’m pretty sure we squashed the nasty ‘white-screen of death’ in Lighthouse. It turns out it had to do with the way I had things configured on the server. The great folks at Engineyard helped me track down the error message after pouring through a bunch of log files, calling strace on processes, etc. It’s been fixed, so things should keep running smoothly now.

If you care about a more technical response, it had to do with the way I was rotating logs. I was using the ruby Logger library from a snippet of code I’ve carried with me for awhile. It’s from my days of deploying on Textdrive and getting nasty grams about constantly growing log files. As you may have gathered, I’m used to more experienced system administrators handling the apps I write. Engineyard has been a huge help for me in keeping Lighthouse running this long, with clustered web servers, memcached, background job processing.

One thing that has come out of this, is a better monit configuration. Monit is a tool that keeps tabs on the running application processes and restarts them if needed. We recently added URLs that monit will periodically access directly too to ensure that the application is working fine. Finally, I set up a script to email me when any of the processes die. Even monit wasn’t able to properly detect the various ‘white-screen’ issues we were having…

Discussion

Glenn Gillen said on January 11th

We’ve moved to God instead of monit for a range or reasons, most importantly though it’s flexibility and detailed logging of everything it’s doing. http://god.rubyforge.org
rick said on January 13th

I asked Ezra about it several months ago while monit was being setup, but Engineyard seems to prefer monit. I think they are looking for other solutions though.

Sorry, comments are closed for this article.

Subscribe to Tumblelog and Articles. Subscribe to Articles only.

Active Reload is a small company founded by Rick Olson and Justin Palmer. Rick is a member of the Ruby on Rails core team, while Justin is a member of the Prototype Javascript Library core team. Together we build rocking software with a focus on awesomeness.

Beautifully simple issue tracking. With a beautiful Mac-like interface, email and subversion integration, Lighthouse is the perfect project management tool. We took the suck out of issue tracking.
Web-based Subversion Interface Manage Subversion permissions and browse through your changesets and files with ease. Warehouse takes the complicated and mundane tasks of Subversion and makes it easy.
We've changed the way you manage your blog. With a beautiful interface, multi-site support, and infinitely extendible through plugins, Mephisto is a fresh take on what a blogging application should be.

May29

permalink

twitter bot

Read your tweets for messages prefixed with @foo and post to that user. http://pastie.caboo.se/65927

# Read tyler-durden's tweets for any messages starting with '@project-mayhem'
#
#   $ ruby twitter_bot.rb tyler-durden project-mayhem my-password
#
# Stores the last modified and highest id in a yaml file in the pwd by the anme of 'twitter_bot.cache'
# Should block against dupes and be easy against twitter's servers

May17

permalink

Displaying nice relative dates

<%= relative_time(Time.now) %>
# today
<%= relative_time(1.day.ago) %>
# yesterday
<%= relative_time(1.day.from_now) %>
# tomorrow
<%= relative_time_span([Time.now, 5.days.from_now]) %>
# May 17th - 22nd

— relative_time_helpers rails plugin

May16

permalink

Discussing a Feature Removal

Rick:  no more lightbox?
Justin:  no, I banished it because it was useless the way it was being used.  
Rick:  ha, it was cool man
Justin:  yeah, Zune music sharing is cool too.
Rick:  touche

May10

Scrobbler: Last.fm for Ruby —Ruby Library for accessing the AudioScrobbler web service.
permalink