Recent Posts by John Leach

Topic: Brightbox Help / Rails 3.1 support

We now have 1.9.3 packages with Passenger support too:

http://wiki.brightbox.co.uk/docs:ruby-ng

Topic: Passenger / libapache2-mod-passenger deb and ruby1.9.x

Hi Kristian,

We now have some “next generation” ruby and passenger Ubuntu packages that support ruby 1.9.3 – you can find the details here: http://wiki.brightbox.co.uk/docs:ruby-ng

John.

Topic: Brightbox Help / Rails 3.1 support

Hi Ali,

yeah, we’ve added rack 1.3.2 packages which is needed to use 3.1 with our passenger packages. For 19.2, try out our ruby-ng ppa:

apt-add-repository ppa:brightbox/ruby-ng

it provides ruby 1.9.2 packages. It has a few quirks though atm – I’d recommend using a server in fully 1.8 or fully 1.9.2 “mode”, as mixing the two versions ends up causing some issues atm.

Also, our passenger packages don’t yet support 1.9.2 – you’ll have to use the gem version of passenger with that.

And the version of ruby 1.8 in that repository is not Ruby Enterprise enabled, just stock Ruby.

So a few niggles, but they’re usable. We’re still working on them and will officially support them as soon as we’re happy with them (soon!).

John.

Topic: Brightbox Help / SPF Records

Hi Chris,

mail sent through our relays should never go via any host on the kcom.com domain, or that IP (neither of which are associated with us at all), so I don’t know what is going on there. Our senders list is up to date. Make sure your app is configured to use some 3rd party relay of some kind.

John.

Topic: Brightbox Help / Domain Name Change Problem

Hi Nikki,

I can’t think how adding a new domain name would have caused this. Did you edit your Apache config manually, or did you use our Capistrano recipe?

John.

Topic: isitruby19.com / Comment spam

I think we’ve cracked the spam problem now. Defensio.com is blocking the majority of it now, with no false positives so far.

John.

Topic: isitruby19.com / Comment spam

Hi Roger, Dave,

thanks for the heads up on this and sorry for not responding sooner. We’ve been manually cleaning up the spam for a while now but it’s relentless. We were rather naive in thinking the captcha would prevent spam on it’s own (though it did ok for a while!).

Anyway, we’re now working on adding proper spam checking, using the defensio.com service. It’s actually been deployed already but we’re still seeing spam so it’s clearly still work in progress. I’d hope we’ll have it sorted next week and the code on github.

John.

Topic: Brightbox Help / Deployment and apt-get password

Hi,

the password is needed to run sudo commands and sudo doesn’t support ssh keys.

You can tell Capistrano the password in the recipe though, and it’ll enter it for you whenever it’s needed:

set :password, “mysecrets”

we do recommend keeping this out of your code repository though – see the advanced gem settings page on the wiki for example of how to do this:

http://wiki.brightbox.co.uk/docs:gemv2:advanced…

John.

Topic: Brightbox Help / Imagemagick hogging CPU

Hi Jim,

this might be due to a corrupt jpg that was uploaded, or maybe a bug in imagemagick.

alternatively, it might be simply that you’re running very low on ram and there isn’t quite enough for imagemagick to do its magick.

stopping it hogging so much cpu is tricky. You can use rlimits (see the ulimit command in the bash manpage) but that will just result in it being killed if if uses more then x seconds of cpu.

John.

Topic: Brightbox Help / SMTP Relay

Hi Brett,

open a support ticket with us with some details of a missing message and we’ll track it down in our relay logs and see where the problem might be.

John.

Topic: Brightbox Help / deployment problems

Hi,

this is a bug in Capistrano’s windows support. Have you tried using the very latest capistrano, 2.5.17? See if that fixes your local deployment problem.

If this still isn’t fixed, as rob5 said, a workaround is to use git or svn deployment – not the “local” deploy (which uses the xcopy command and gets it wrong). Put your code in an svn or git repository and deploy from that to your server.

John.

Topic: Brightbox Help / deploy via github

Hi Volker,

github have a guide on deploying from github using Capistrano:

http://github.com/guides/deploying-with-capistrano

that might help you confirm a few options.

We use an ssh-agent to avoid having to put private keys on our servers. You load your key (or the github deploy key) into your local ssh agent, and then when git connects to github over ssh, it tunnels the authentication back to your desktop machine (see the Capistrano option mentioned in that article, ssh_options[:forward_agent] = true)

Hope that helps!

John.

Topic: Passenger / Ruby Enterprise 1.8.7-2009.10 Packages for Ubuntu Hardy

Rahoul,

no problem, reverting back to the stable REE packages is just like the procedure to revert back to the standard Ubuntu packages, but with a slightly different apt preferences file.

Just change the apt source from rubyee-testing back to rubyee, and add the following lines to /etc/apt/preferences

Package: libopenssl-ruby1.8 libreadline-ruby1.8 libruby1.8 ruby1.8 ruby1.8-dev irb1.8 rdoc1.8 ri1.8
Pin: release C=rubyee
Pin-Priority: 600

Then do an apt-get update && apt-get upgrade. The old packages will be listed as downgrades.

John.

Topic: Brightbox Help / Odd MySQL results with ORDER BY and indexes

Hi Ian,

ah yes, I’ve managed to reproduce it. Looks like a MySQL bug – I can’t find the exact bug reported, but some similar ones. We’ll be upgrading MySQL as part of some routine maintenance in a few weeks, so this may be fixed then.

The reason adding the extra condition on cemetery_id fixes it is because MySQL then chooses to use the cemetery_id index. I’d recommend removing the firm_name and surname indexes for now (perhaps replace them with one multi-column index (known as a composite index) and see if that helps)

John.

Topic: Passenger / Ruby Enterprise 1.8.7-2009.10 Packages for Ubuntu Hardy

We’ve built 32bit and 64bit Ubuntu Hardy packages for Ruby Enterprise 1.8.7-2009.10. More details are on the blog. If you have any trouble or want to report success, please post in this forum. Thanks!

Thanks!

Topic: Passenger / passenger, how many mysql connection per instance

hi Volker,

Passenger should only open one mysql connection per passenger instance. This was the same with mongrel – mongrel didn’t open 2 per mongrel.

The only special case is during a restart. If you do a “soft” restart, by touching restart.txt, passenger may start new instances before the old ones have finished their work and been shut down. In this case you could end up with up to double the number of MySQL connections in use – but only for a very short period. It used to be worse with old versions of Passenger as passenger instances would hand around until their idle expiry, but now it shuts them down when they’re finished with their current request.

Our gem has a setting for passenger restarts, :hard or :soft. :soft touches restart.txt, so you could end up with too many instances for a moment. :hard will kill the processes directly – this stops you having too many instances, but could interrupt a long running request.

See the wiki for more information on this: http://wiki.brightbox.co.uk/docs:gemv2:passenge…

John.

Topic: Brightbox Help / Odd MySQL results with ORDER BY and indexes

Hi Ian,

I’ve tried this query on your db and it seems to be ok now. The query returns 3,002 rows with or without the surname. I can see that the table still has the indexes, so I’m assuming this was another problem of some kind that you’ve since solved? It’s not something I’ve ever seen MySQL do before – how did you solve it?

As a side note, MySql can actually usually only use one index per table per query, so you usually need a composite (multi-column) index. But in this particular case it’s able to do an index merge so you’re fine (you can use the EXPLAIN command to see what indexes MySQL will use for a particular query).

Thanks,

John.

Topic: Passenger / Passenger 2.2.5 + fixed NGINX packages

I’ve just pushed i386 and amd64 packages for passenger 2.2.5 to our apt repositories.

I also updated the nginx-brightbox package in the testing repository and fixed the path problems (and added the upload module too).

Topic: Brightbox Help / trouble installing ruby1.8-dev

Hi Mike,

you need to install our version of ruby1.8-dev as well then – that is available from our repository too.

John.

Topic: Brightbox Help / LDAP logins

Hi Gareth,

I can’t think of any problems. As long as both the rails and bbox-admin accounts are still accessible then it should be fine. We wouldn’t officially support it without further discussion though, so best not done on gold supported boxes.

John.

Topic: Brightbox Help / Unplanned reboots

Hi,

it’s certainly not to be expected, no. If there are any physical server issues, we’re alerted to them very quickly and we update our status blog with the details – there was nothing going on this morning for sure.

So, something is going wrong inside your Brightbox. If an app consumes too much ram on your Brightbox, it can affect other processes and even hang the box. There is usually evidence of this in the kernel log, though sometimes it can hang the box quicker than it can log anything. Try:

grep “Out of memory” /var/log/kern.log /var/log/kern.log.0

That will show you logs of processes being killed, this would be a clue something is using too much ram.

The default passenger settings allow up to 6 app instances to be running in total – if these are rails apps, this would be a bit of a challenge for a 256 box (especially if you have more than one application set up). You’ll need to tweak the settings a bit to limit the damage on a small box. I’m working on a Passenger tweaking guide at the moment which should help with this (it should have been done this week but it’s been delayed a bit – expect it next week on the blog).

Hope that helps,

John.

Topic: Brightbox Help / using global_shared_dirs for network shares

Hi Adrian,

our gem does have the concept of a globally shared directory (as you say, something like NFS so multiple boxes have access to the same files) but it does not provide the sharing it self, it only sets up the symlinks. It defaults to the same behavior as local_shared_dirs. The reason it has support for both types is that some files, like logs, we don’t recommend storing on global storage but you still want them shared between releases of the app on the same box.

Yes, you’d need to setup something like NFS – though using NFS would mean you’d have a central point of failure in your cluster.

We provide a managed shared filesystem product using a clustered filesystem called GlusterFS. This allows, amongst other things, mirroring of files to multiple boxes for high availability. If you’d like more info on this service, just open a support ticket about it.

I just had a quick look at the Ubuntu instructions on NFS – they look ok, but it’s not something we can officially support.

Thanks,

John.

Topic: Brightbox Help / unable to load iptables

Hi Alexis,

it’s likely that you need to install the extra kernel modules package, so iptables can load what it needs. See http://wiki.brightbox.co.uk/docs:kernel#extra_k…

Thanks,

John.

Topic: Brightbox Help / kernel upgrades

Hi Mark,

the kernel is set by our systems, upgrading it from within your machine won’t make any difference – so you’ve done no damage.

To clarify, Brightboxes are 32bit userland, but with 64bit kernels. That is why the kernel says x86_64 but file says libraries are 32bit LSB relocatable. This allows us to use a fast stable 64bit kernel, but not have Ruby (and other processes) waste RAM dealing with 64bit pointers.

Our managed MySQL boxes are 64bit userland, as MySQL benefits from being 64bit more than Ruby/Rails does.

Topic: Brightbox Help / Server is slow or unresponsive (internal server error)

For most common Rails deployments, swap space just delays the inevitable ram exhaustion, and in the process slows down your box massively and puts very heavy load on our SANs. So it is disabled by default. We recommend that you plan your deployments so that you don’t overcommit your ram.

I think 8 Rails web apps on a 256 box is stretching it too far. Even if your apps were very very small (let’s say 35mb), and you limited them to 1 instance per app, that would still be 280mb RAM, even before Apache, ssh etc. Even with the highest Passenger spawner settings, I doubt you’d get 8 apps running on there. And you only get 5 MySQL connections with a 256 box too, so only 5 instances could use the MySQL server at a time (assuming they need it).

I’d recommend you estimate how much ram each app usually uses after it’s been running a while, and then upgrade your Brightbox accordingly. To ensure the least ram usage, set PassengerMaxInstancesPerApp to 1 and PassengerMaxPoolSize to the number of apps you need to have running. Also, try setting RailsSpawnMethod to smart (though this can break some apps, so test thoroughly).

More documentation on these settings here:

http://www.modrails.com/documentation/Users%20g…

Topic: Brightbox Help / Server is slow or unresponsive (internal server error)

also the output from:

sudo grep "Out of memory" /var/log/kern.log

Thanks,

John.

Topic: Brightbox Help / Server is slow or unresponsive (internal server error)

Hi Harianto,

can you paste any relevant look messages from /var/log/apache/error.log ?

Thanks,

John.

Topic: isitruby19.com / JRuby gems?

Is it JRuby has been launched by Carl Mercier – it’s a fork of our project:

http://blog.brightbox.co.uk/posts/is-it-jruby

Topic: Passenger / Could not find RubyGem passenger (>= 0) (Gem::LoadError)

I’ve just had to fix this problem for a customer and thought it’s worth sharing it here.

The customer was using our Passenger Ubuntu packages but their app wouldn’t start and just dumped a 500 errors.

Looking in /var/log/apache/error.log I saw:

/usr/lib/ruby/1.8/rubygems.rb:636:in `report_activate_error': Could not find RubyGem passenger (>= 0) (Gem::LoadError)
	from /usr/lib/ruby/1.8/rubygems.rb:141:in `activate'
	from /usr/lib/ruby/1.8/rubygems.rb:49:in `gem'
	from /usr/bin/passenger-spawn-server:18

After investigation I found that the customer had installed the passenger gem before using our packages and it was interfering. They’d uninstalled it but it seemed to have left some files behind.

I did this to fix it:

$ dpkg --purge passenger-common libapache2-mod-passenger

$ rm /etc/apache2/mods-available/passenger.load /etc/apache2/mods-available/passenger.conf /etc/passenger_version.txt

$ rm /usr/bin/passenger-*

$ apt-get install libapache2-mod-passenger

$ a2enmod passenger

$ /etc/init.d/apache2 restart

Hope that helps others.

John.

Topic: Brightbox Help / Don't know how to build task 'db:create'

Hi David,

assuming you’re using our deployment gem, it tries to run db:create which isn’t in Rails 1.1.6. Our gem creates a rake task file when you first run it on an application which defines some dummy tasks if it’s an old version of rails. You need to make sure this rake file is committed to your scm repository.

John.