Running your own email PUSH server in 3 easy steps

**borked** · 11-28-2012, 08:30 AM

Note, you should download zpush from source and not svn as that howto suggests, as z-push 2 is much better.

**Vapid - BANNED FOR LIFE** · 11-28-2012, 09:38 AM

I do email marketing.

**JamesM** · 11-28-2012, 09:57 AM

i still remember back in days service provider used to charge additional charges for push email service , and i was using pop3 via phone.

**MaDalton** · 11-28-2012, 09:58 AM

have gmail pick up your pop3 accounts and use imap...

**borked** · 11-28-2012, 10:33 AM

Originally posted by MaDalton

have gmail pick up your pop3 accounts and use imap...

imap isn't the same as push - more like:
have gmail pick up you pop3 accounts, and have your server push gmail email to your iphone...

honestly, push email has nothing to do with imap. Once the server receives it, you receive it (+/- 5 seconds). With imap, you are restricted to fetch, which I had configured for every 30 mins. This way, the battery life is far better, since the phone doesn't have to fire up a 3G connection to fetch email.

Been running it for about 5 days now and the phone life during the day before would run from 100% to ~50ish% with no other activity. Now it rests around 85-90% if I don't do any other activity.

**sheken** · 11-28-2012, 11:04 AM

Originally posted by borked

imap isn't the same as push - more like:
have gmail pick up you pop3 accounts, and have your server push gmail email to your iphone...

honestly, push email has nothing to do with imap. Once the server receives it, you receive it (+/- 5 seconds). With imap, you are restricted to fetch, which I had configured for every 30 mins. This way, the battery life is far better, since the phone doesn't have to fire up a 3G connection to fetch email.

Been running it for about 5 days now and the phone life during the day before would run from 100% to ~50ish% with no other activity. Now it rests around 85-90% if I don't do any other activity.

I think you are grossly uninformed/misinformed

First of all, the benefits of Exchange are that it can sync contacts & calendar as well. This solution only syncs up e-mail... kind of useless.

There is only one provider out there that can do true Push and that is RIM through Blackberry. They have the patents at the carrier level where the tower will physically notify the device when there is stuff to receive. The device doesn't "check in" for new data.

Since nobody else can do this, the other players came up with various ways to try and do the same, but with big drawbacks.

IMAP IDLE basically keeps a connection open all the time in IDLE mode until the server notifies the client that new mail has arrived. Although the client doesn't check every X minutes, the connection is still open and it drains the battery.

Exchange Account / Google Sync ( Activesync ) does the same exact thing. It keeps a connection open at all times thus also draining the battery.

Conclusion: Get a Blackberry if you are serious about e-mail. I hear version 10 will rock.

Otherwise, just use IMAP IDLE. Makes no sense to configure z-push only for e-mails, without contacts & calendars.

**sandman!** · 11-28-2012, 11:19 AM

http://blackberry.com/

**MaDalton** · 11-28-2012, 11:30 AM

Originally posted by borked

imap isn't the same as push - more like:
have gmail pick up you pop3 accounts, and have your server push gmail email to your iphone...

honestly, push email has nothing to do with imap. Once the server receives it, you receive it (+/- 5 seconds). With imap, you are restricted to fetch, which I had configured for every 30 mins. This way, the battery life is far better, since the phone doesn't have to fire up a 3G connection to fetch email.

Been running it for about 5 days now and the phone life during the day before would run from 100% to ~50ish% with no other activity. Now it rests around 85-90% if I don't do any other activity.

imap i only mentioned for convenience - i just recently learned the benefits

but gmail seems to do push mail - i havent set any frequency and i am getting notified for each mail i receive

**borked** · 11-28-2012, 11:35 AM

Originally posted by sheken

I think you are grossly uninformed/misinformed

I think you are outdated with your info.

Push email is just that - it pushes the email to the sync'd device immediately. Yes a ping is required to maintain that connection (Blackberry/RIM or other), but I'm telling you that battery life per day with fetch set to 30 mins was drastic compared to push email on the same device configured for push and not fetch.

And yes, I have my cal synced through the exact same z-push server, thanks to owncloud - but that isn't the point of the thread...

Have you tried IMAP IDLE on an iPhone or blackberry on 3G? Obviously not due to your comments.... it is *far* from being a "poor man's push" service...

If you think Blackberry holds the patent on push email, give this a read:
http://en.wikipedia.org/wiki/Push_email
then get back to me, because what is written in the RIM section is *exactly* what z-phone does.

**borked** · 11-28-2012, 11:36 AM

Originally posted by MaDalton

but gmail seems to do push mail - i havent set any frequency and i am getting notified for each mail i receive

I have gmail imap set on my phone but I never got it pushed...
did you try sending yourself an email from your desktop to see if you receive it on your phone within 20 secs?

**RyuLion** · 11-28-2012, 11:42 AM

Great..more spam..

**borked** · 11-28-2012, 11:50 AM

Originally posted by RyuLion

Great..more spam..

Dude, I can look at my server email logs live and see easily 1 email a second getting rejected, yet I only get like 1 or 2 max spam emails a day slipping through.

And more importantly, and this is the absolute truth, I'm yet to receive anything suggesting a false positive got binned....

A well configured email server works wonders

---edit
ok maybe not per second, but not far off

Code:

Nov 28 21:52:13  spamdyke[7956]: DENIED_RDNS_MISSING from: [email protected] to: xxxxxxxxxxxxxxxx origin_ip: 212.63.115.23 origin_rdns: (unknown) auth: (unknown) encryption: (none) reason: (empty)
Nov 28 21:52:18  spamdyke[7958]: DENIED_RDNS_RESOLVE from: [email protected] to: xxxxxxxxxxxxxxxx origin_ip: 187.240.134.189 origin_rdns: customer-tolu-134-189.megared.net.mx auth: (unknown) encryption: (none) reason: (empty)
Nov 28 21:52:23  spamdyke[7962]: DENIED_IP_IN_CC_RDNS from: [email protected] to: xxxxxxxxxxxxxxxx origin_ip: 187.146.237.250 origin_rdns: dsl-187-146-237-250-dyn.prod-infinitum.com.mx auth: (unknown) encryption: (none) reason: (empty)
Nov 28 21:52:29  spamdyke[7983]: DENIED_RDNS_RESOLVE from: [email protected] to: xxxxxxxxxxxxxxxx origin_ip: 85.94.115.245 origin_rdns: adsl-lns1-l5108.crnagora.net auth: (unknown) encryption: (none) reason: (empty)
Nov 28 21:52:33  spamdyke[7989]: DENIED_IP_IN_CC_RDNS from: [email protected] to: xxxxxxxxxxxxxxxx origin_ip: 37.15.196.44 origin_rdns: 44.196.15.37.dynamic.jazztel.es auth: (unknown) encryption: (none) reason: (empty)
Nov 28 21:52:36  spamdyke[7992]: DENIED_RBL_MATCH from: [email protected] to: xxxxxxxxxxxxxxxx origin_ip: 88.6.134.61 origin_rdns: 61.red-88-6-134.staticip.rima-tde.net auth: (unknown) encryption: (none) reason: 
Nov 28 21:52:40  spamdyke[7994]: DENIED_IP_IN_CC_RDNS from: [email protected] to: xxxxxxxxxxxxxxxx origin_ip: 217.94.22.179 origin_rdns: pd95e16b3.dip0.t-ipconnect.de auth: (unknown) encryption: (none) reason: (empty)
Nov 28 21:52:43  spamdyke[7998]: DENIED_RBL_MATCH from: [email protected] to: xxxxxxxxxxxxxxxx origin_ip: 217.42.39.70 origin_rdns: host217-42-39-70.range217-42.btcentralplus.com auth: (unknown) encryption: (none) reason:

**sheken** · 11-28-2012, 11:55 AM

Originally posted by borked

I think you are outdated with your info.

Push email is just that - it pushes the email to the sync'd device immediately. Yes a ping is required to maintain that connection (Blackberry/RIM or other), but I'm telling you that battery life per day with fetch set to 30 mins was drastic compared to push email on the same device configured for push and not fetch.

And yes, I have my cal synced through the exact same z-push server, thanks to owncloud - but that isn't the point of the thread...

Have you tried IMAP IDLE on an iPhone or blackberry on 3G? Obviously not due to your comments.... it is *far* from being a "poor man's push" service...

If you think Blackberry holds the patent on push email, give this a read:
http://en.wikipedia.org/wiki/Push_email
then get back to me, because what is written in the RIM section is *exactly* what z-phone does.

And you get your information from wikipedia ? Come on... Every car has an engine and 4 wheels, doesn't mean they are the same. Everyone can claim they do push, but nobody does push like RIM. They are the real deal. Everyone else has weaker methods. That's why a Blackberry's battery can last for days and not die.

Here it is explained in more detail than that Wikipedia article says. They don't even define "push" properly.

The BES monitors the email server, and when it sees new email for a BlackBerry user, it retrieves (pulls) a copy and then pushes it to the BlackBerry handheld device over the wireless network.

With Activesync the device maintains a connection to the Exchange/Z-push server and gets notified when a new mail arrives.

With BES, the BES server polls the inbox, forwards the message to RIM's Mail Hub in Canada/US which then contacts your carrier which in turn truly pushes the message from the tower to the device.

Activesync "push" is just a marketing gimmick.

Second of all, you are doing something wrong or seeing some skewed results somehow. There is no logical reason for an application that connects one time every 20 minutes to check mail, to take up more power than an application that maintains a connection open all the time. In fact, on my S3 I keep it on check every 5 minutes because "push" really kills my battery. And yes, I run z-push as well with Calendar, Contacts & Tasks synced up.

**borked** · 11-28-2012, 11:57 AM

From the comments above, it's clear that people being connected *live* via email on their mobile device isn't that important, but I do server admin stuff where monitors on the servers and clients email me when something goes awry, so having an email come through *live* is really important for me.
Hosting providers must be similar, so hope they can appreciate this thread. YMMV and probably you couldn't care less! Anyways, I'm not one to hold back useful stuff, so take it or leave it ;)

**borked** · 11-28-2012, 12:07 PM

Go on, I'll bite...

Originally posted by sheken

Here it is explained in more detail than that Wikipedia article says. They don't even define "push" properly.
+++
The BES monitors the email server, and when it sees new email for a BlackBerry user, it retrieves (pulls) a copy and then pushes it to the BlackBerry handheld device over the wireless network.
+++
With Activesync the device maintains a connection to the Exchange/Z-push server and gets notified when a new mail arrives.

Riiiight, and what is this active monitoring of the email server? A ping perhaps?
Oh, guess what - Exchange email does just that - if the device hasn't heard a ping from the server in the last X minutes (configurable), it pings the server, because that means the IP address of the client has changed. How is this different, from RIM's almighty offering?

Originally posted by sheken

With BES, the BES server polls the inbox, forwards the message to RIM's Mail Hub in Canada/US which then contacts your carrier which in turn truly pushes the message from the tower to the device.

Activesync "push" is just a marketing gimmick.

hahaha - you haven't looked into z-push have you?
It is exactly that - it polls the imap server every X seconds (I have it configured to 5 seconds, cos my server is beefy and can handle all that shit), and then when a state change is observed (ie email received) it recouperates it and pings the client who grabs it.

How is this different from your description of a true push server?

Originally posted by sheken

Second of all, you are doing something wrong or seeing some skewed results somehow. There is no logical reason for an application that connects one time every 20 minutes to check mail, to take up more power than an application that maintains a connection open all the time. In fact, on my S3 I keep it on check every 5 minutes because true push really kills my battery. And yes, I run z-push as well with Calendar, Contacts & Tasks synced up.

Firstly, it takes a lot of power (relatively speaking) to fire up the 3G connection, locate the tower, make the connection, then check (no email) power down, than it does to maintain an active 3G connection. If I was on a train for example, then push would be shit and as bad as pull as the phone would always be polling to try to find a tower, but outside of that example (or flying along a motorway), you will always be polling a tower from your provider nearby, so no searching needed.

**MaDalton** · 11-28-2012, 12:10 PM

Originally posted by borked

I have gmail imap set on my phone but I never got it pushed...
did you try sending yourself an email from your desktop to see if you receive it on your phone within 20 secs?

less than 20 sec

**borked** · 11-28-2012, 12:12 PM

Originally posted by MaDalton

less than 20 sec

IMAP IDLE then... your cell is on 3G or WiFi? In my experience IMAP IDLE works great on a wifi, but never works on a 3G connection (tested by looking at raw email logs live)

But, gmail is ever evolving, so maybe they do push now. If so, that's damn cool

**MaDalton** · 11-28-2012, 12:13 PM

Originally posted by borked

IMAP IDLE then... your cell is on 3G or WiFi? In my experience IMAP IDLE works great on a wifi, but never works on a 3G connection (tested by looking at raw email logs live)

But, gmail is ever evolving, so maybe they do push now. If so, that's damn cool

it definitely works on 3g the same way - i am running this since a week now

**borked** · 11-28-2012, 12:14 PM

and yup,
http://email.about.com/od/iphonemail...Phone_Mail.htm
looks like they do. Unfortunately, as sheken points out, it's not real push, cos only RIM can do that....

**borked** · 11-28-2012, 12:15 PM

MaD - can I interest you in owncloud?
;)
I hate my shit being stored on 3rd party servers!

**MaDalton** · 11-28-2012, 12:20 PM

Originally posted by borked

MaD - can I interest you in owncloud?
;)
I hate my shit being stored on 3rd party servers!

i have just set up everything so it works flawlessly on my laptop, phone and tablet...

and once you surrender to Google it all works great

**sheken** · 11-28-2012, 12:22 PM

Originally posted by borked

Go on, I'll bite...

Riiiight, and what is this active monitoring of the email server? A ping perhaps?
Oh, guess what - Exchange email does just that - if the device hasn't heard a ping from the server in the last X minutes (configurable), it pings the server, because that means the IP address of the client has changed. How is this different, from RIM's almighty offering?

hahaha - you haven't looked into z-push have you?
It is exactly that - it polls the imap server every X seconds (I have it configured to 5 seconds, cos my server is beefy and can handle all that shit), and then when a state change is observed (ie email received) it recouperates it and pings the client who grabs it.

How is this different from your description of a true push server?

Firstly, it takes a lot of power (relatively speaking) to fire up the 3G connection, locate the tower, make the connection, then check (no email) power down, than it does to maintain an active 3G connection. If I was on a train for example, then push would be shit and as bad as pull as the phone would always be polling to try to find a tower, but outside of that example (or flying along a motorway), you will always be polling a tower from your provider nearby, so no searching needed.

Dood, are you serious ? You claim to be some sort of server admin but can't seem to grasp a few simple concepts.

First of all, BES server and Exchange server are usually on the same INTERNAL LAN. That means that the cost of BES server polling Exchange even every second is close to 0, as opposed to your device polling Exchange through the airwaves every X seconds.

Second of all, I don't even know how much more clear I can make this. Activesync maintains an open connection to Exchange/Z-Push, Blackberry doesn't.

Maintaining an open connection at all times, isn't really push.
Having the tower contact your device and pushing information to it, IS push. The difference between these two methods is why Blackberries have a much better battery life.

End of story. Jeezus

**borked** · 11-28-2012, 12:24 PM

I doubt I'll ever surrender - too much satisfaction in doing it myself and knowing I have full control :p

**ottopottomouse** · 11-28-2012, 12:24 PM

Google seems to spit emails at my phone pretty much straight away (android gmail app).

**borked** · 11-28-2012, 12:33 PM

Originally posted by sheken

Maintaining an open connection at all times, isn't really push.
Having the tower contact your device and pushing information to it, IS push. The difference between these two methods is why Blackberries have a much better battery life.

End of story. Jeezus

So let's clear this because you seem to be comparing "true push" to SMS messaging ?
If so, is apple's iMessage not a tru push protocol?
There's something you are missing and that is what is defined as 'live polling'

**borked** · 11-28-2012, 12:36 PM

Originally posted by ottopottomouse

Google seems to spit emails at my phone pretty much straight away (android gmail app).

Yeah, MaD confirmed this that i didnt know.
Still, my email doesnt pass via gmail so im naive.
In any case this is all part of a more cunning plan.... Due to this a learnt about Gamin libs... And Baldrick knows all about gamin for his cunning plan....

**sheken** · 11-28-2012, 12:53 PM

Originally posted by borked

So let's clear this because you seem to be comparing "true push" to SMS messaging ?
If so, is apple's iMessage not a tru push protocol?
There's something you are missing and that is what is defined as 'live polling'

I think you are finally starting to understand it. Read on PIN messaging from RIM.

In SMS terms think of it this way.

BES knows there's an e-mail waiting for the user. He forwards the e-mail to RIM. RIM talks to their carrier partners and they send an SMS to the device saying: hey, there's an e-mail waiting for you. connect and get it.

The device connects to the server and gets the message. No other connection is necessary. Do you understand know how this is true push ?

**borked** · 11-28-2012, 01:03 PM

Originally posted by sheken

I think you are finally starting to understand it. Read on PIN messaging from RIM.

In SMS terms think of it this way.

BES knows there's an e-mail waiting for the user. He forwards the e-mail to RIM. RIM talks to their carrier partners and they send an SMS to the device saying: hey, there's an e-mail waiting for you. connect and get it.

The device connects to the server and gets the message. No other connection is necessary. Do you understand know how this is true push ?

No, I think you are the one that is still not getting it....
The principal is exactly the same - BES uses the Server Routing Protocol, which is proprietary to them (probably what you are alluding to), but its principal is still exactly the same... as the blackberry must be connected to the the BB infrastructure (the connection) then the push is still the same - a simple ping to tell the device a message is waiting, so go grab it.
The only difference is the BES device is connected to the RIM infrastructure, rather than my iphone connected to my server.

--edit
SRP communication takes place on TCP port 3101 by default. SRP work in two different part: 1. SRP ID 2. SRP Authorization Key

My iPhone uses TCP port 443. My iPhone works in two parts: 1. my login ID 2. my password

--end edit

It's still TCP. No difference.

It has nothing to do with the SMS protocol, dummy.

**bDok** · 11-28-2012, 01:23 PM

Originally posted by borked

IMAP IDLE then... your cell is on 3G or WiFi? In my experience IMAP IDLE works great on a wifi, but never works on a 3G connection (tested by looking at raw email logs live)

But, gmail is ever evolving, so maybe they do push now. If so, that's damn cool

For an iphone on gmail you can configure with the exchange server. You connect to m.google.com with your account for the exchange server.

For me I have a bunch of calendars. To start this will only connect your main account calendar. For the rest you have to on your phone go to m.google.com/sync in web browser and turn on the others.

Now you have contact syncing, email, and calendars through gmail.

**sheken** · 11-28-2012, 01:24 PM

Originally posted by borked

No, I think you are the one that is still not getting it....
The principal is exactly the same - BES uses the Server Routing Protocol, which is proprietary to them (probably what you are alluding to), but its principal is still exactly the same... as the blackberry must be connected to the the BB infrastructure (the connection) then the push is still the same - a simple ping to tell the device a message is waiting, so go grab it.
The only difference is the BES device is connected to the RIM infrastructure, rather than my iphone connected to my server.

--edit
SRP communication takes place on TCP port 3101 by default. SRP work in two different part: 1. SRP ID 2. SRP Authorization Key

My iPhone uses TCP port 443. My iPhone works in two parts: 1. my login ID 2. my password

--end edit

It's still TCP. No difference.

It has nothing to do with the SMS protocol, dummy.

ugh... do you even understand what I'm telling you ? Ofcourse it gets the e-mail through TCP. It's a data connection.

The difference in 'PUSH' is whether:

1. You maintain an active connection at all times.
2. You get notified by the cell tower ( through something similar to an SMS ) that you are to open up that TCP connection.

#1 IS NOT push, #2 IS push.

Why do you think RIM paid 612 million dollars to NTP back in 2006 after patent litigation ? Because the idea of opening a connection to a server hadn't been thought of by anybody else ?

Anyhoo, I'm done wasting time on this. Read up on it if you really want to learn

**borked** · 11-28-2012, 01:27 PM

Hahahaha - ok, the penny just dropped... the difference between BES and BIS.
If what you just said is true, you are indeed a gullible person!

Enjoy your RIM version 10 on your enterprise server!!!

**borked** · 11-28-2012, 01:29 PM

== Blackberry Enterprise Service
vs
Blackberry Internet Service

lol!!

--edit
even more lols

Great table!

**borked** · 11-28-2012, 01:31 PM

Originally posted by MaDalton

it definitely works on 3g the same way - i am running this since a week now

Originally posted by bDok

For an iphone on gmail you can configure with the exchange server. You connect to m.google.com with your account for the exchange server.

For me I have a bunch of calendars. To start this will only connect your main account calendar. For the rest you have to on your phone go to m.google.com/sync in web browser and turn on the others.

Now you have contact syncing, email, and calendars through gmail.

yes yes, that's all been dealt with - gmail now supports P-IMAP and apparently it is out-of-the-box for mobile clients (from what otto and MaD said above)...

**sheken** · 11-28-2012, 01:35 PM

Originally posted by borked

Hahahaha - ok, the penny just dropped... the difference between BES and BIS.
If what you just said is true, you are indeed a gullible person!

Enjoy your RIM version 10 on your enterprise server!!!

what are you talking about ? BES is Enterprise, BIS is Personal.

Have you even bothered to read up on the subject ?

http://gizmodo.com/5331151/giz-explains-how-push-works

Beyond this, go read the tech docs from RIM.

And for the record, I switched this summer from a Bold to an S3 and I'm apalled at how shitty the battery is. I'm going back to Blackberry 10 when it comes out.

**borked** · 11-28-2012, 01:38 PM

Originally posted by sheken

what are you talking about ? BES is Enterprise, BIS is Personal.

Have you even bothered to read up on the subject ?

http://gizmodo.com/5331151/giz-explains-how-push-works

Beyond this, go read the tech docs from RIM.

And for the record, I switched this summer from a Bold to an S3 and I'm apalled at how shitty the battery is. I'm going back to Blackberry 10 when it comes out.

hehe, yup, I'm pissing myself actually.
I didn't expect to get baited in a simple HowTo thread that only concerned people that were interested in it, but you decided to bait me, and I learned a lot. And I can simply say, that my thread still stands. Maybe RIM have something good with their proprietary BES way of doing things, but shit what a price to pay!!!

Meanwhile, us plebs will carry on with a no-good push service, gmail or home built and mearly look on in awe

**sheken** · 11-28-2012, 01:42 PM

Well, as long as you learned something.

**borked** · 11-28-2012, 01:43 PM

Originally posted by sheken

Well, as long as you learned something.

I'm glad this thread doubled your post count... maybe after 10 years, so did you

**sheken** · 11-28-2012, 01:45 PM

ha. I have better things to do than to post here, but from time to time I do get involved

**borked** · 11-28-2012, 01:48 PM

Hmmmmm jumping into a thread so quickly.... Methinks you use another nik for your regular posts...
Unless of course you really do browse gfy each hour for the last 10yrs without posting!!

**PSD** · 11-30-2012, 12:16 PM

Originally posted by borked

I have gmail imap set on my phone but I never got it pushed...
did you try sending yourself an email from your desktop to see if you receive it on your phone within 20 secs?

You need to set it up as a exchange account, that said, if you are using an ios device, exchange accounts are currently having problems with ios 6.0.1

Edit: Nevermind, missed bDok's post but actually gmail uses exchange not p-imap

**borked** · 12-19-2012, 11:31 PM

Originally posted by JCK

Edit: Nevermind, missed bDok's post but actually gmail uses exchange not p-imap

Google is set to remove this feature at the end of January
Source

Google last week said it's closing Google Sync, a service that allowed users of Microsoft’s Exchange ActiveSync protocol to access their Gmail, calendars, and contacts via the Microsoft Exchange ActiveSync protocol.

The service will now only be continued for paying users of Google’s email and collaboration-ware, with Google employing the open-standards CalDAV, CardDAV and IMAP for the free services.

So no push email from gmail next year

**PSD** · 12-20-2012, 12:01 AM

Interesting, if you set up your device before 1-30-13 they will continue to support it.

http://support.google.com/a/bin/answ...answer=2716936

Running your own email PUSH server in 3 easy steps

Running your own email PUSH server in 3 easy steps

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment

Comment