From time to time here at Mars Hill Church we receive emails with questions about the work that we do, who does it, what we use to do it and how the magic happens. For the second installment lets look at the website that kicked off the current sermon series, askanything.marshillchurch.org
Q. I’d love to get some background on how you developed the Ask Anything app - a very nice app, by the way. Was it all custom?
A. The short answer is “yes it was custom, thanks”.
The big idea was taken from the letter to the Corinthians, wherein Paul uses his letter to answer some popular questions from the church body in Corinth. We wanted to take a few months to collect questions and then take a one Sunday each to preach and answer the top 9 questions.
Development and design was an all done in-house here where we knew generally what we wanted out of it, though we also let the needs evolve as we developed it, and we could get things going quickly and adapt as need, so it ended up being coding in Ruby on Rails.
We looked into some more off the self type solutions, such as ShouldDoThis from the great Robot Co-op guys here in Seattle, but they often did not offer us enough control over the presentation nor the content posted by users, and any kind of phased approach. Though we wanted it to remain as open and as organic as possible, knowing we were opening this up to the world, we planned to need some level of control over things such as vulgarity, personal attacks and completely off topic questions. We did want our admin involvement to be as minimal as possible though.
Going into it there were a few key things we knew we wanted:
- We only wanted positive feedback, so we did not go with voting things up and down, we only wanted people to be able to vote FOR something, not against it.
- We wanted it to have a very low barrier of entry for people to ask questions. So we decided to not have a login. Which we knew meant we could not control gaming the system very easily, but we kind of liked that.
- Although we wanted it to be easy to add questions, we still wanted good questions. So we decided to limit the amount of characters that could be used to 200. It was a somewhat arbitrary number, but the point was to make people think about the heart of the question and not ramble.
- We wanted a broad audience, so it should be easy to share the question you asked with your friends. So we made sure each question had a landing page with a URL that could be blogged or given to friends and encourage them to go vote.
- We wanted it to be sticky, interesting and as fair as possible. So we opted for 10 votes a day and a basic comment section for each question. Then later added a small blogging section and RSS features.
- We wanted some self moderation by users. So we allowed flagging of questions and comments by anyone. These went to a queue and were reviewed by an admin . If a flagged item was approved then it could no longer be flagged. We later also added the ability for users to suggest a similar questions that should be merged.
Overall the site worked great, and was a success. We ended up with doing 4 phases.
Phase 1- Open voting, where anyone could post any question onto the site and vote 10 times a day on any question.
Phase 2- Site closed to new questions. We stopped taking questions about 2 months into it with almost 900 questions asked. Then allowed a month of continued voting on all questions.
Phase 3- We reduced the available questions to the top 50 and let voting continue. During this phase we also merged similar questions, based on user suggestions. To do this we combined votes from similar questions and if needed we reworded the question to include any content from merged question as well, we tried to minimize those changes though and leave the questions as intact as possible.
Phase 4- Initially we had not planned on it but it became apparent that we needed to limit the 50 questions and focus attention on the most popular so with about 20 days to the end we removed all but the top 20 questions. Many of the lower number questions at that point need too many votes to be viable for th top 9.
What did we learn:
- Where people can cheat they will. We spent a good amount of time watching and deleting votes from people who voted thousands of times a day. Also there is really no fool-proof way to watch IP addresses and sessions and cookies to be 100% accurate without a login.
- People have a lot of time to debate theology and talk trash to other Christians on anonymous forums. I often thought we needed to offer an apology to some peoples employers.
- We love Ruby on Rails. It was confirmed that we like developing in this and is now where we are investing our development efforts for most new projects.
Let us know if you have any more questions and we will try and answer them in the comments.