Blue Duck Station Stray Staff Trip

Blue Duck Station is such a wicked place. Just spent two nights down there with the Stray and Spaceships team and my awesome girlfriend Monique.

The view from top of the waterfall:

More videos are at the bottom!

Bottom of the waterfall:

Waterfall Kayak:

Kayaking:

Horse Riding:

Quad Ride:

Sheep Dip:

Serve files to guests phones at your event via Wi-Fi

The problem: you want to give a free MP3 audio file to your guests at your event and would like something more scalable, faster, cheaper and easier than giving out CDs and/or loading USB sticks manually, and would prefer the file to end on their smartphone anyhow where they can listen on headphones quickly.

Solution: Serve the files via Wifi with a locally hosted laptop web server on a private wireless network (WLAN) with no access to the internet required.

Users download mp3 or similar file to their iPhone/Android/laptop from the free wireless network. No internet connection is needed and download is full wire speed.

Users see's the following QR Code for http://freetune.funk.co.nz/ to scan:

Visit http://freetune.funk.co.nz/ on your phone for MP3 Download

Plan:

• Laptop (in our case a Mac) running:
  • Locally modified hosts file with entry for freetune.funk.co.nz pointing to localhost (in my case 10.0.1.1)
  • Apache setup to serve the file with a local website running on the laptop
  • Apache is redirecting all requests to http://freetune.funk.co.nz/ which is only available on the private LAN.
  • Sharing it's non-existent "internet" via DHCP over the ethernet port
• Wi-Fi router (I'll use a WRT54GL linux router) set to:
  • Open network no password (or publish password inside event)
  • Laptop is connected to as the upstream WLAN ethernet port and is default gateway
  • Clients phones and laptops connect to this router
  • Hopefully the router can handle ~200 clients connecting to it through the evening
• Business card printouts containing a QR Code for http://freetune.funk.co.nz/
• Instructions for users of iOS and Android to scan the code or visit http://freetune.funk.co.nz/
• Backup real DNS entry
  • on live internet for your private LAN IP (say 10.0.1.1 or 192.168.1.1)
  • Done to that in case someones phone can get a real DNS server it will still resolve to 10.0.1.1
• Captive Portal idea:
  • Ideally an extra tweak to DNS make every lookup for any domain go to this IP address
  • Apache setup to redirect all requests to the private IP

Laptop web-server will be plugged into the "Internet" port, left side

Get rid of live.com load-balanced domains with a regular expression: GA Webmail Referral Traffic Source Rollup Filter

The problem? Nasty load balanced domains in Google Analytics reports like this:

The solution? Clean nicely segmented source lines that "roll up" into one:

To clean these up requires two filters:

• Webmail Source Rollup (search and replace with Webmail (brand])
• Webmail Medium Rollup (swap / referral with / email as medium for further rolling up!)

Both these filters will use the core regex code I've figured out that consolidates 99% of the worlds webmail systems without pulling in any false positives in theory.

The Magic Regex Code

Here is the filter regex that I've been currently using in my production Google analytics advanced filters since 15 Feb 2012 to cleanup these - or to roll up the load-balanced domains that you often get in referrals:

My starting point:

(messag|courrier|zimbra|imp|mail)(.*)\.(.*)\..{2,4}

This grabs any domain with say "mail" in it, but runs a check on the ending of the domain: it needs to have at least 2 dots after it and a TLD between 2 and 4 chars long. It will miss go.mail.ru and mail.com. It will also miss "Mail Campaign \ email" because this is not a proper domain. So far so good 🙂

My improvement with exceptions (This is the one to use!):

(messag|courrier|zimbra|^imp|mail).*\.(.*\..{2,4}|go\.mail\.ru|promail\.co\.nz|service\.mail\.com|3c\.web\.de|outlook\.com)

Domain of webmail platform is in capture group $A2.

My improved regex has a new exception section at the end to allow some special cases (go.mail.ru, 3c.web.de, service.mail.com) to get through the filter using a hard-coded approach that skips the safety net and autonomy provided by the wide open keyword matching paired with a domain name restriction ensuring "two more dots and a TLD".

Maybe that is excessive use of regex, but at least you can be sure you can now see your word of mouth / word of email traffic nice and tidy!

How does it work?

I'll break the formula down in sections:

(messag|courrier|zimbra|^imp|mail)

This looks for the really obvious and common keywords in webmail services, the main one being mail. This will match sn124w.snt124.mail.live.com but it will also grab emailchimp.com and a huge number of others that you really don't want to catch with this filter. If we were looking for live (but we aren't in this case), then a site like www.answeringoLIVEr.com would get picked up in the crossfire. I use a ^ in front of imp so that domains like dimpost.wordpress.com don't get caught.

Basically the first part of the domain name (sn124w.snt124.mail) will be getting deleted by this filter so you could stand to lose quite a lot of data with the "mail" and "imp" keywords if this were the only parts of the expression! So the next bit of filter is designed to pass the domain through another difficult test involving the dots and TLDs...

(.*)\.(.*)\..{2,4}

This makes sure that the domain bits after mail or zimbra or whatever always have two dots and a TLD (top level domain extension eg .nz .jp). Which matches the end bit of sn124w.snt124.mail.live.com and  the end of: www.funk.co.nz.

The (.*) part means match anything including nothing, and the \. means there must be a dot, so (.*)\.(.*)\. means there gotta be at least two more dots in this domain name coming up after the live". Which is how Answering Oliver gets through the test for live. The next part .{2,4} is all about the top level domain or TLD. These can be 2, 3, or 4 letters long like .co, .com. and .mobi. The curly braces specify how many times the previous character . (which means single char you like except nothing) can appear like {min,max}.

Then in the middle is a pipe | which cuts the regex open and allows some really hard to match exceptions through for smaller webmail systems. Only reason you see the web.de one also appear on the left of the central | is because this is such a whacky domain name that it doesn't match the "mail" which gets most of the webmail systems on the planet. Germans aye? 🙂

* (Hi Devon! Thanks for sending traffic to Stray Travel I found you researching this post)

Screenshots

Campaign Source Filter

Advanced filter.

Field A -> Extract A: Campaign Source:

(messag|courrier|zimbra|^imp|mail).*\.(.*\..{2,4}|go\.mail\.ru|promail\.co\.nz|service\.mail\.com|3c\.web\.de|outlook\.com)

Field B -> Extract B: [leave both blank]

Output To -> Constructor: Campaign Source: Webmail ($A2.$A3)

GA Webmail Rollup Filter

Medium Rollup Filter

Advanced filter.

Field A -> Extract A: Campaign Source:

(messag|courrier|zimbra|^imp|mail).*\.(.*\..{2,4}|go\.mail\.ru|promail\.co\.nz|service\.mail\.com|3c\.web\.de|outlook\.com)

Field B -> Extract B: [leave both blank]

Output To -> Constructor: Campaign Medium: email

GA Webmail Rollup Filter

Additional reference domains to check:

The domains below are the really rare webmail clients that are hard to extract:

mail175-236.sinamail.sina.com.cn
go.mail.ru
service.mail.com
promail.co.nz
3d.web.de
ch1prd0310.outlook.com

This is why I needed to grab the full domain with (.*\..{2,4}) versus the first versions (.*)\.(.*)\..{2,4} which would have only grabbed "sinamail". Now we get  sinamail.sina.com.cn.

These can be checked with: sina\.com\.cn|go\.mail\.ru|mail\.com|promail\.co\.nz|web\.de|outlook\.com

Errors and Issues

Currently the filter is incorrectly tracking as word of email the following referrals:

dailymail.co.uk

I will update the filter to address this issue at some point.

References

Thanks to Olivier Resoneo for the original inspiration (French). His code was:

Grouper tous les webmail francophones sous le nom de domaine principal

Custom filter

Advanced

Champ A : Campaign Source : (messag|courrie|zimbra|ima?p|mail|prd[0-9]+)(.*)\.(.*)\..{2,4}

Champ B : (rien) -

Output To -> Constructor : Campaign Source : Webmail - $A3

Yes

No

Yes

No

On peut aussi décliner pour forcer le medium à 'email' quand match sur Campaign Source, ET Campaign à email-non-taggue par exemple, pour avoir le triplet medium/source/campagne

MegaUpload

I'm in Coatesville bitch!

Update: Excellent interview with Kim Dotcom on Campbell Live.

Now I had never heard of MegaUpload until this story broke in NZ about it, and him living in Coatesville mansion (that place looks wicked!!) which is in my region (Auckland)! Straight up, that's intense!

But I feel sorry for him, I hope the pussy NZ government doesn't bend over and extradite him (they will) I think it sounds a bit of an over-reaction at this point in time, I mean take this excerpt from Wikipedia below, I have yet to find anything decent about exactly what he's done wrong. He claims he's innocent according to reuters. Perhaps he is bad, even if he is, taking the site down should be enough, he doesn't need to be arrested and extradited.

Down she goes... poof!

The best article I've found so far has been this Ars Technica one Why the feds smashed MegaUpload. At least this makes me confident now the feds have some "proof" knew they were pirates, but so is YouTube in many ways too, and at least both YouTube AND MegaUpload provide a take-down tools for copyrights holders to remove stuff. Another good place to check is this Quora page.

Scariest part of it all tho, is this bit at the end of the NYT article: “The government hopes to use their private words against them,” Mr. Kerr said. “This should scare the owners and operators of similar sites.”

Legal commentators point out that while the indictment may be correct and Megaupload might have acted as a criminal conspiracy as claimed, a number of points in the indictment are based upon selective interpretations and legal concepts (described in one article as "novel theories" of the law) and could be challenged in court. An L.A. Times analysis stated that the author was "struck by how far the indictment goes to find something nefarious"; likewise a TechDirt analysis concluded that while the founder of Megaupload had a significant history of "flouting the law", evidence has potentially been taken out of context or misrepresented and could "come back to haunt other online services who are providing perfectly legitimate services". Both analyses concur that other evidence could show criminality; the concerns were not irrefutable.

I mean the site's been replaced by an FBI logo, stink. They reckon at one stage is was the 13th biggest site on the net.

Anonymous Respond

Global hacker group anonymous have responded by creating this web browser based DDOS tool:

http://pastehtml.com/view/blakyjwbi.html

...which seems to automatically fire up a bunch of connections to justice.gov. Nasty!

Created by: Paralegal.net

On tour in NZ with Stray

Day One - Ha Hei / Hot Water Beach
Somehow - my mistake - I didn't realise the bus would actually pick up at Ponsonby Backpackers, which is right around the corner from my house, and so could have slept 30 mins longer and saved myself a cab ride to town.

Got myself a seat next to a lovely girl from Munich in German and learnt how to count to ten. Made it to Ha Hei and Hot Water beach, such a beautiful place.

Day Two - Raglan
Im writing this at the end of day two of the stray tour. Were in Raglan at Karioi lodge. It's the meck shit - a real backpacker hostel in the jungle, with a really hot sauna and big lounge. Dope as aged rims tree sits smack in front of me over the deck, and the hostel is cool . The lodges are situated nestled into a valley in from the road on the south head.

Apart from a few teething problems early on involving changes to the schedule and wales breaking in the bus in Thames in true form (who needs a rear window anyhow? nice fresh air!), the tour is exceptionally cool still. Great bunch of people, heaps of Germans, plus a regular mix of French, English, Irish, Dutch, Italian, Brazilian and me.... the sole kiwi in group. In fact im a staff members trying to masquerade as a true backpacker.
Surfing turned out to be a highlight so far - that and nearly burning my feat at hot water beach briefly on our trip at Ha The on the first night. We hired some boards from raglan surf school at the beach at ### (where the raglan reggae festival was) and shit was pretty mellow. I can see why pros would go further out at the other beach.

Taupo friday night
Had a big night on friday in taupo. Good times and crazy dancing with Jordon, Nick, Anders, Avril, Veronica, and... gets hard to remember all the names.

Blue Duck
Pariatua loop walk: Awesome old growth trees with long lichen hanging off and inquisitive tiny smaller than normal fantail-looking birds come to say, while pairs of tuis pull aerobatic manoeuvres. Steep terrain. Lords of the rings type shit, made me feel like the hobbit.

National Park
Did the tongariro crossing! 19.4kms, 800m up, and 1100m down. Very cool 7 hour walk! Amazing endorphin rushes after the walk. Spa after was fun, that's when I taught the Germans what a sausage fest was (worst fest in German).

Bring back the Google Calendar Facebook export feature

UPDATE: This feature is now *working correctly* again, hooray for Facebook and Google! I wonder if my blog post helped get this fixed?

A really handy feature I used all the time to escalate important Facebook events in my Google Calendar has just been brutally and cruelly taken away, so I'm gonna blog about it and start a petition to try and get it back.

I used to do the following after clicking an RSVP to an event on Facebook: Click Export -> Send Email and it would send a nicely formatted email like in the screenshot below that contains:

• An Add to calendar >> link formatted to magically work well with Google Calendar
• The actual name and date of the event (hello?)

As you can see (well I can anyhow cos I'm a geek and can read URL-endocded data) this means that with one click you get a fully formatted Google Calendar entry after clicking (2nd screenshot)

Sin Rave - Done the old way that rocked my world with convenience

And after clicking "Add to calendar" you get this if you're logged into Google like I always am:

It *used to* create calendar entry automatically

And now

 ... and now the sad reality is that Facebook have changed it so it looks like this:

Perhaps not many people used this feature, and I know that Facebook recently did a bunch of changes to reduce the amount of emails their system is spewing about the place, but these emails were really good. The main change was the removal of the URL encoded calendar item.

Here is a copy of the SIN item for reference:

http://www.google.com/calendar/event?action=TEMPLATE&pprop=eidmsgid%3A_ckp3ac9k60r38d1o68p3echj6906cob3clh6urrb5phmur8_13343e5749edd332&dates=20111111T230000%2F20111112T070000&text=SIN+%5B11%2F11%2F11%5D&location=373+KRD+%28420+%26+RISING+SUN%29+&details=MAKE+LOVE+NOT+WAR++++++++++++%0A++%0AUpstairs%3A+ARENA+1+++++++%0A%E2%97%8F+Soundsystems%3A+Demolition+%2B+In+House+%0A%E2%97%8F+Styles%3A+House%2C+Dnb%2C+UK+Funky+%26+Dubstep+%0A%E2%97%8F+Burlesque+by+La+Ragdoll+De+Bon+Bon+%26+Ladi+Sin+Quana%0A%E2%97%8F+Mini+Fashion+Show%0A%E2%97%8F+Armistice+Themed+Strip+Show+%283+Ladies+for+3+Hours%2C+Full+Strip%29%0A%5BStrip+show+from%3A+11%3A11%3A11+11%2F11%2F11%5D++%0A%E2%97%8F+Breakdancers%3A+B+Boy+Discover+%2B+TBA+%0A%E2%97%8F+Rockstar+Girls+X+2+%0A%E2%97%8F+Lighting%3A+Lazers%2C+strobes%2C+black+lights%0A%E2%97%8F+Atmos%3A+Smoke%2C+Bubbles%2C+Visuals%0A%0APart+1%3A+%28Bounce+and+Stare%29%0A10%3A00-10%3A50%3EJUSTIN+SANE%0A10%3A50-12

So according to this thread:

http://www.google.com/support/forum/p/Calendar/thread?tid=3b76e6c18640db63&hl=en

It is not even possible to set Google Chrome as the default handler for the .ics file extension that Facebook wants me to download now. Turns out I have to following these laborious steps for each event import that used to take 1 click:

To import events from iCalendar or CSV files, just follow these steps:

1. Click the down-arrow next to Other calendars
2. Select Import calendar
3. Click Choose file and find the file that contains your events, then click Open.
4. Select the Google Calendar where you'd like to import events, then click Import.

That's quite a few extra steps since it also involves actually opening Google Calendar, and putting your mouse over the cal you want to import into. Sucky.