Over the past few days I’ve been looking into a new search engine for an application. The chosen search daemon is sphinx and it’ll be called via the Java API (that comes with sphinx) from ColdFusion. Getting the basic searching going was pretty easy, creating an index, do a basic search, set some filters, done. The next step was to look into geodist matching, calcuating how far apart two latitude / longitude’s are, slightly more tricky. In CF you can do it with some crazy curvature of the earth calculations but in sphinx it’s real easy – when you know what it wants. The key there is radians.

You’ll find plenty of tidbits on the web about this but no real end to end examples so hopefully this will help!

If you need to geocode some postcode or zip values I can highly recommend tinygeocoder.com. It’s basic and gives you a straight lat / long value which is perfect. If you want something with a bit more information you’ll need to check out the Yahoo! YDN geocode API or Google’s HTTP API. I’ve got the YDN working with ColdFusion (I’ll post about that later).

First up you’ll need some data, I’ve created a basic DB table called sphinx_test.sql which you can download. It’s been written for MySQL. Quick note on the float values, if you don’t specific float 10,6 (or something similar) your lat/longs will automatically be chopped to only 4 decimal places.

The SQL contains 3 locations in England. The British Airways London Eye (51.502893,-0.118811), Millennium Dome (51.501984,0.004764) and Windsor Castle (51.481971,-0.600686).

Once you’ve created your table you’re ready to create an index. I’m going to assume you’ve already installed sphinx in /opt/sphinx (this is on a CentOS platform). To get the Java API part for ColdFusion is easy as well.

[root@hosting ~]# cd /root/sphinx-0.9.9-rc2/api/java
[root@hosting ~]# make
[root@hosting ~]# cp sphinxapi.jar /opt/coldfusion8/runtime/servers/lib
[root@hosting ~]# /opt/coldfusion8/bin/coldfusion restart

This is the sphinx.conf you’ll need to get this running. Now there’s a whole load of stuff you could do in the configure file but I’ll just give you a basic one which should be enough.

Note you have two options when it comes to getting the lat/long from MySQL, you can either use RADIANS(val) or use a pre-stored the radian value of the lat / long In this case we’re going to use the latter but I’ve included a commented line for the former.

Now for the actual code that does the work. I’ve created a file called sphinx.cfm (view as text or click it to actually see it working) with comments inline. The interesting parts are:

<!--- sort order must be @geodist to allow distance matching --->
<cfset variables.sphinx.SetSortMode(variables.sphinx.SPH_SORT_EXTENDED, '@geodist desc')>

<!--- where are we starting, this will be the london eye latlong converted to radians --->
<cfset variables.sphinx.SetGeoAnchor('latitude', 'longitude', degreesToRadians(51.502893), degreesToRadians(-0.118811))>


All of the API calls are the same as the documented PHP functions so you can refer to the documentation for more information.

Thanks go out to various bloggers and Tim to get this running.

The continued, and if by continued I mean actually starting, migration of my sites over the Rackspace Cloud is moving forward again at last. I’ve nearly finished setting up the initial Cloud Servers and all the DNS. The 4th server setup this evening however shows more signs of the move from Mosso to Rackspace Cloud, the inital RDNS entry for the previous 3 had been slicehost.com addresses but this time out it’s 000-000-000-000.static.cloud-ips.com. A quick lookup shows that the domain still lives on slicehost nameservers and has a Rackspace contact but I guess it’s movement.

Some changes are present in the control panel but not a lot, maybe they’ll be more or maybe it is just a logo update.

Update:

So after some playing I’m not going anywhere. The cloud servers are great, real easy to get running but for what I’m doing and have hosted, mess around with etc a dedicated box is ideal. Currently with The Planet and have a good deal, a UK server would be nice but the cost just doesn’t make it viable right now. I might look at either a bigger TP box, more RAM or go to iWeb or ServerBeach but not sure – any recommendations or reviews of those?

I’ve finally got resin, railo and caucho playing together in a nice, no longer ripping each other’s hair, out kind of way. I have to say my biggest comment (and probably one of the more tricky things to do) is that for Railo to become a big player in trying to take away Adobe’s CF user base installers are going to be essential. That said railo does seem quicker on processing CFML (need to do more tests to verify that).

I’ve got apache 2.2.3 running on CentOS 5.3. I have railo 3.1.0.015 (not updated to 016 yet) and caucho.

My resin.conf has this addition (multiple times for the different domains):

<host id="www.domain.co.uk" root-directory="/home/domain/public_html">
<host-name>www.domain.co.uk</host-name>
<host-alias>domain.co.uk</host-alias>
<web-app id="/" document-directory="."/>
</host>

The apache virtual hosts are just normal vhosts. No extras.

My caucho.conf (in /etc/httpd/conf.d) looks like this:

LoadModule caucho_module /usr/lib64/httpd/modules/mod_caucho.so
ResinConfigServer localhost 6800
CauchoConfigCacheDirectory /tmp
CauchoStatus yes
<Location /caucho-status>
SetHandler caucho-status
</Location>

Hopefully that might help someone out. I’m going to try and do a from scratch CentOS build guide for The Rackspace Cloud sometime next week. I’ve started migrating some sites over to the cloud server and so far not hit any issues.

Some people aren’t aware that for Rackspace Cloud Servers they also offer DNS which is pretty handy. The setup however is a little odd and the interface different to any other DNS/zone editor I’ve used. There’s a number of KB entries under The Rackspace Cloud control panel as well.

A quick guide on how to enter the details to get it to work. If you login, access your server and click DNS you’ll first need to add the domain name, in these examples domain.com. Once done click on the domain and then you can add away. You can’t edit, well if you call delete and add editing then yes you can.

So, for an A record:

Name: domain.com
Content: 1.2.3.4
Priority: 10
TTL: 300
Type: A

And a CNAME record:

Name: cname.domain.com
Content: another.domain.co.uk
Priority: 10
TTL: 300
Type: CNAME

For sub-domains you have to put the full domain in rather than just the sub-domain so:

Name: sub.domain.com
Content: 1.2.3.4 (or a CNAME)
Priority: 10
TTL: 300
Type: A

For MX records they say just use your IP as the content:

Name: domain.com
Content: 1.2.3.4
Priority: 10 (increment for additional MX entries)
TTL: 300
Type: MX

I think however you can also do this:

Name: domain.com
Content: 10 mail.domain.com (assuming you have a standard A record setup with IP called mail)
Priority: 10
TTL: 300
Type: MX

That’s it.

I’ve decided to take a leap and try moving all the sites I host into the cloud. Currently I’ve got a dedicated server but it doesn’t really do a lot most of the time and the pay for usage idea appeals. I’ve got a cloud server over at Mosso (soon to be The Rackspace Cloud) running CentOS 5.3 and I’m also in the process of getting Railo server / Resin running to handle the CFML stuff. So far it’s going OK.

Few things that I needed to get resin to compile, like gcc, glibc-devel, openssl-devel, openssl but aside from that pretty easy. I’ll try and write up a step by step guide when it’s working.

Posting this more as a reminder to myself than anything else, but, if you need to replace all instances of the word old in a file with the word new this command is really handy!

sed -i 's/old/new/g' file.txt

Currently looking into a whole load of performance related issues with an application at work. Basically, how to make it go faster and support more users online at any given time.

Currently our setup has a MySQL 5.0.x master running with quite a few slaves. The setup is OK but isn’t as scalable as we’d like. The master isn’t suffering load wise so we don’t need to look at distrubtion masters or get into pyramid replication which is just a bit too complex. The main hang up is searching. We have members, each member has loads of related details, photos, payments, messages, profile information etc and currently it’s spread out over a number of tables. We’ve looked at sphinx, hadoop, solr / lucence but aren’t sure if those kinds of things can be used for multi faceted objects. We don’t want a plain text search because the data is mostly lots of numbers like height, age, hair colour all picked from lookup tables but maybe we can combine our data into some form of index that is better for searching? Currently we flatten the most searched data out into single tables (split on certain categorizations) and search those the details in them are updated regularly by triggered actions in the application. Memcache is an option as we use it to store data already.

MySQL 5.4 looks great in the performance enhancements it could bring but that’s now not looking like it’s going to be coming until around December. I hope Sun release it sooner however as the inital peek at it we’ve had seem to back up Sun’s claims.

Any ideas / suggestions / experience would be appreciated. Comment or email me.

As you may or may not know I work at Global Personals / WhiteLabelDating.com. We’re currently after some top quality Ruby on Rails Developers to work from our Windsor office here in the UK. If you think you might fit the bill drop Ross Williams a line (rwilliams@globalpersonals.co.uk) and be sure to mention you saw the advert here.

Job Description

Global Personals, the company behind WhiteLabelDating.com, has a fantastic opportunity for talented, experienced Rails developers to join our development team.

You’ll join 11 developers, designers and project managers focused on creating the best online dating platform in the industry. We’re just beginning a series of new projects in Rails – you’ll be joining right at the start to help influence decisions based on your expert Rails skills and experience.

You’ll be based on the top floor of our lovely headquarters at 1 High Street in the middle of Windsor. It’s easily commutable from London (West via Paddington or South-West direct from Waterloo), Reading, Bracknell, Staines or Slough.

We give our people the best tools. You’ll get a brand new 24in iMac and a 23in Cinema Display to work on (or a top-end PC if you insist) and you’ll sit at a big desk on a swanky Hermann Miller Aeron chair. We have a free posh coffee machine in our own kitchen, XBox 360, Wii, LCD TVs, Rock Band, Guitar Hero, Mario Kart, football table and other toys.

Salary will be based on experience, but will likely be in the range of £35k-£60k (negotiable).

We pay a £1000 bonus after your three month probation if you’ve come direct to us. Following your successful probation we’ll pay up to 5% of your salary into your company pension, contribute to gym membership, plus other benefits.

You’ll need the following skills/attributes:

• Ruby on Rails – essential to demonstrate your Rails experience
• MySQL (or equivalent RDBMS)
• Strong communicator, teamworker and best practice stickler
• Excellent knowledge of Javascript using Prototype/Scriptaculous
• Ability to write clean, semantic, well-formed HTML and CSS
• Proficiency with Linux or Unix
• Subversion

The following are desirables:

• TDD and/or BDD
• Experience of Agile working environment
• Open source projects or blogs you’ve written on
• Previous dating or social networking projects
• ColdFusion and PHP

Company Description

The largest private online dating company in the UK with offices in Windsor, Miami, New York and Sydney.

Adobe have today released announced the 2nd hotfix for ColdFusion 8.0.1. You can go get it from the Adobe site. It has quite a few bug fixes listed as well.

The World Wide Web as we know it turns 20 tomorrow, 13th March 2009. The following is a brief history of what went on (source unknown).

Tomorrow (Friday), the European Organisation for Nuclear Research (Cern) will celebrate the 20th birthday of the worldwide web, an event that sparked a technology revolution.

Although the two words are widely thought to be interchangeable, it is important to distinguish the web from the internet, which has existed in various forms since the 1950s. The internet defines a series of interconnected networks that allow data packets to travel, these days mostly using the TCP/IP transmission protocol, from one connected system to another.

The worldwide web, on the other hand, is more a network of content servers that allows information to be shared in specially formatted documents, hosted on systems connected to each other via the internet.

From humble beginnings as a proposal for a more effective document management system for internal use by Cern personnel, the web has grown exponentially into a global phenomenon, where almost a quarter of the world’s population convene to view and publish every type of information imaginable, and as a platform for communication, commercial transactions, dating, e-learning, entertainment, file swapping, software distribution and many other activities too numerous and diverse to list here.

So much so, that 20 years on, our reliance on the web for all, or the vast majority, of our information gathering has now become so complete that major internet outages that deny us access can quite literally bring our world to a crashing halt.

Cern has chosen Friday 13 March to mark the 20th anniversary of the day when one of its employees, Briton Tim Berners-Lee, first proposed a “universal linked information system” that would over the next few years develop into what we now know as the worldwide web. It was originally designed as a way of improving information management within Cern  then as now a large organisation employing several thousand people who all needed to communicate, share information, equipment and software. Read more…