Is spam email still a big deal? Does it really consume that much bandwidth? I can imagine in the past when computers were slower that spam was more of a resource hog than it is now.
Much of the email I now receive is spam. This is both because of and a reason why I've abandoned using email -- it's too painful and too low-reward to even open my inbox. Privacy, surveillance, and other concerns are leading me to largely abandon email (and most other private electronic comms) in any regard.
If you want to set up your own email service, addressing span, and proving you're not a spammer yourself is a major hassle. There are hoops to jump through on both ends, reducing the incoming flow, and ensuring that outbound peers will pick up your mail.
If you're engaged in any large-scale outreach (think: accounts-based services, notifications, etc.), then dealing with spam accusations on your own service are an issue. Those might or might not be legitimate -- in many cases, users will, with a high degree of legitimacy, not try to unsubscribe or opt out of email but instead simply flag it as spam. As accounts die or are closed, simply continuing to email them may mark you as a spammer (a periodic re-validation process is a good way to avoid this). I've seen decade-plus old services with hundreds of thousands or millions of questionable addresses. The devs may not care about that, but as ops, and as the point person for ensuring that notifications and alerts do get out, I do.
The noise alone of spam, outbound or inbound, causes logistical problems.
Thanks for your thoughtful response. What is the future of email? How will person to person text communications evolve so that they are more secure, in your opinion.
That's hard to call, though I see a few dynamics affecting the outcome.
1. Cost matters. In communications, a low- or zero-marginal-cost solution almost always has an advantage overy any marginally-priced solution. I'm quite confident in predicting that micropayments will not be the solution here, though some time-based all-you-can-eat solution might work, more likely at an annual or decadal scale than a monthly (or shorter) one.
2. New media tend to be adopted based on attractive founding cohorts. People are drawn to communities that are selective, exclusive, and/or generally represent a higher socio-economic-political status than they have. If you look at the history of communications, going back to speech and writing themselves, those with facilities in spoken or written language were seen as attractive and appealing.
Early Usenet, and later Facebook, both developed among selective-admissions research/prestige universities. I don't think this is at all a coincidence, and each network could be contrasted with more general, less-selective emerging or extant networks of the time, say, BBSes, Prodigy, AOL, or MySpace. See dana boyd who's also pointed this out.
(Corollary: do you want to create a Facebook killer? Create a billion-dollar fund and and seed $10m to each of 100 new networks within an aspirational community, reserving the right to merge any successful networks emerging from those into an aggregate, aspirational community. Google came close to this formula for G+, but erred, contrary to a great deal of commentary, in opening up the network too soon, and most especially, to the kiss-of-death native Google community of marketers, SEO, and advertising. If it had stuck to tech and STEM for a few years, it might have created that vital community. In my opinion.)
3. The directory is a critical element. In SMTP email, this includes DNS and local elements. DNS itself is a stumbling block for many people to self-hosting services, and it really wasn't meant to scale to billions of individual domains. There are hierarchical structers that could do this, including DNS hierarchies (domain, subdomain(s), host, user), or X-500 or LDAP, but which have not been widely adopted. Microsoft's Active Directory (an LDAP variant) exists, but is proprietary. I'm not sure what the solution here looks like, but it's actually among the key sticking points.
4. Support and privacy of metadata. The history of the standard RFC 822 message header structure is an interesting one, and dates to the early-20th century development of the inter-office memo. See Kathy Yates, "The Memorandum as Management Genre" (http://www.ismlab.usf.edu/dcom/Ch6_YatesMemoMgtCommQtly1989....). Among the surprises I found there, "In-reply-to" actually pre-dates" the "From", "To", and "Subject" fields -- creating chains of referenceable communications matters.* A key problem of current SMTP messaging is that these metadata are among the most useful to any surveillance system, capitalist or state, and yet are not protected by extant content-encryption systems. At the same time, some access to metadata may be critical for filtering purposes.
5. Various forms of trust and graduated trust-delegation. This applies at various levels. There's a level of trust which must generally be granted to message transport systems -- software, hardware, channels, protocols. Filtering against spam or unwanted email may involve some level of disclosure of either identity or contents. (It's likely that spam won't generally be (strongly) encrypted making filtering of it more viable, but this remains to be seen.) The ability to layer trust and encryption such that access, approval, and/or vouching may occur at various levels in a limited fashion could help.
6. Built-in full whitelist / greylist / blacklist capabilities. This is almost certainly going to be required generally of networked infrastructure, not just email. As an example: I'm finding I am making increased uses of extensive blocklists for general network and web traffic. Mechanisms for doing this on a generally straightforward, useable, trustable, and effective basis will almost certainly be required. This includes the ability to selectively-though-intentionally bypass such filters on an occasional basis.
The ability to restrict general messaging access to known trusted and vouched sources, to provide for bypass on an occasional basis, to allow third-party vetting (and to specify the trust/distrust level of same), etc., all strike me as necessary. Keep in mind that this applies to any messaging and comms channel, inclusive of postal mail and voice telecoms, both of which are also increasingly subject to nuisance attacks. I see the imminent death of conventional phone systems within a few years if the junk call problem isn't addressed, and most likely the solution to that being a disposal of number-dial-based systems as a resolution, also within a few years. Already, the abandonment of all phone-based comms by Millennials and many Gen-Xers is fairly widely reported.
7. The alternative, generally, to points 3-6 above, is the emergence of one or more exclusive messaging networks, following generally the selective and aspirational founder cohort as mentioned above. Note again that early comms systems tended to reflect this. Letter-writers, in the early 19th century, reflected the 5-25% of populations which were educated and literate. Telegraphs were limited in access to companies and wealthy individuals (historical note: president U.S. Grant learned of his electoral victory in the home of a neighbour in rural Galena, IL, who had a personal telegraph line installed). Telephones were initially uncommon, found in upscale households, a fact still pervasive enough in 1948 to lead to the famous "Dewey Beats Truman" Chicago Tribune headline based on the misleading sampling error of a teleophone-based survey. Long-distance and international dialing were limited through the 1950s and 1960s respectively. Long-distance charges were high through the 1990s (see above: marginal vs. fixed-rate pricing). Email was initially limited to a restricted set of college undergraduate and graduate students, faculty and staff, government departments, and technology-company professionals. The Blackberry was, for a time, a badge of membership amongst a professional elite.Each of these was a selective and aspirational cohort at its founding. The popularity of the media eventually, in virtually every case, killed the appeal of the addressible community, and did so most especially amongst the highly-valued members.
(One of the recurring themes of science fiction authors' essays of the 1960s - 1980s was the issue of postal mail, and the burdens this imposed on various authors. Arthur C. Clarke wrote on this several times, noting that eventually he'd be reduced to having to send only a pre-printed postcard, "Arthur C. Clarke regrets ...".
Attention is fundamentally limited. Rivality is the counterpoint to virality. The greater the reach of a medium, the more intrusive and annoying it is, axiomatically.
8. Standardisation of formats. I've long strongly favoured simple ASCII email. It's been ... amusing and validating to watch this issue be re-hashed with succeeding generations of messaging systems and devices. HTML allows far too many sins, ASCII not quite enough, but the balance seems to be toward simpler. I'd like to see a semantically structured message format emerge that has the concept of headers, emphasis, possibly even links, and perhaps images, but nothing more. I use the term "STML" (structured text markup language), though there's an extant concept of "POSH" (Plain-old Semantic HTML). The products of simple markup languages such as Markdown would generally work, though experience suggests that any markdown format is too much for the masses.
The ability of recipients to specify what messaging formats they are willing to accept, and have this occur as a negotiation at delivery time, strikes me as especially useful. "This recipient accepts only 7-bit ASCII unencoded messages", for example. Or lists of specifically whitelisted file formats, or encoding formats, by sender or sender category (family, friends, business, vendors, government, strangers, etc.). This serves to impose a complexity constraint on what a sender might attempt to deliver. Yes, you can try to send some highly-complex, highly-formatted newsletter, but 99.98% of recipients will reject it at delivery time, after all other checks have been completed.
Note that standardisation also might include the concept of forms-based email, something that's been ... exceedingly poorly supported to date. This is a concept addressed at length in the Kathy Yates article above about business correspondence standardisation. The ability to send an email that structures responses into a specific format (with, perhaps, a catch-all free-form field required) could be immensely useful.
Standardisation might also include calendar, lightweight messaging, voice/video, and other types of exchanges, all of which current messaging systems ... more or less completely fail at.
9. A re-thinking of authentication and validation. The ability to have the sender's identity be locally verified, and potentially cross-checked against multiple third-party systems could be highly useful. Presently, email quite literally allows anybody to claim to be anyone, simply by filling out the appropriate headers or text. A local identity validation would rely on local PKI validation, whist a third-party validation would send credentials to one or more third-party sites for confirmation. This would tremendously reduce the attack vector for misrepresentation, though of course it also produces risks of metadata leakage -- means for validating without divulging who is requesting the validation would help.
Creating one-off identities triggering validations might be an attack on this though -- the validation could only come from a specific recipient, or someone capable of accessign that party's messages. At the same time, one-off identities and their validation might be used as canaries indicating disclosure of private information, contacts lists, or other data.
Attacking the motives for phishing through improved authentication systems would of course also be useful.
It's a mental resource hog if nothing else. Just changed a setting on my moms phone and noticed she had 108k unread emails between two accounts. All spam.
I got locked out of a Yahoo account years ago that I used solely as a spam account for signups. It was getting hundreds a day and I just looked for that one email confirmation as necessary. Sometimes I wonder if it's still active. It might have a million emails waiting (were Yahoo to allow unlimited messages).
Probably for many people. I haven't even had to think about it in years except to check my spam folders once a week for ham. And that's for 8 email addresses including two catch-alls.
Much of the email I now receive is spam. This is both because of and a reason why I've abandoned using email -- it's too painful and too low-reward to even open my inbox. Privacy, surveillance, and other concerns are leading me to largely abandon email (and most other private electronic comms) in any regard.
If you want to set up your own email service, addressing span, and proving you're not a spammer yourself is a major hassle. There are hoops to jump through on both ends, reducing the incoming flow, and ensuring that outbound peers will pick up your mail.
If you're engaged in any large-scale outreach (think: accounts-based services, notifications, etc.), then dealing with spam accusations on your own service are an issue. Those might or might not be legitimate -- in many cases, users will, with a high degree of legitimacy, not try to unsubscribe or opt out of email but instead simply flag it as spam. As accounts die or are closed, simply continuing to email them may mark you as a spammer (a periodic re-validation process is a good way to avoid this). I've seen decade-plus old services with hundreds of thousands or millions of questionable addresses. The devs may not care about that, but as ops, and as the point person for ensuring that notifications and alerts do get out, I do.
The noise alone of spam, outbound or inbound, causes logistical problems.
TL;DR: Yes.
1. Cost matters. In communications, a low- or zero-marginal-cost solution almost always has an advantage overy any marginally-priced solution. I'm quite confident in predicting that micropayments will not be the solution here, though some time-based all-you-can-eat solution might work, more likely at an annual or decadal scale than a monthly (or shorter) one.
2. New media tend to be adopted based on attractive founding cohorts. People are drawn to communities that are selective, exclusive, and/or generally represent a higher socio-economic-political status than they have. If you look at the history of communications, going back to speech and writing themselves, those with facilities in spoken or written language were seen as attractive and appealing.
Early Usenet, and later Facebook, both developed among selective-admissions research/prestige universities. I don't think this is at all a coincidence, and each network could be contrasted with more general, less-selective emerging or extant networks of the time, say, BBSes, Prodigy, AOL, or MySpace. See dana boyd who's also pointed this out.
(Corollary: do you want to create a Facebook killer? Create a billion-dollar fund and and seed $10m to each of 100 new networks within an aspirational community, reserving the right to merge any successful networks emerging from those into an aggregate, aspirational community. Google came close to this formula for G+, but erred, contrary to a great deal of commentary, in opening up the network too soon, and most especially, to the kiss-of-death native Google community of marketers, SEO, and advertising. If it had stuck to tech and STEM for a few years, it might have created that vital community. In my opinion.)
3. The directory is a critical element. In SMTP email, this includes DNS and local elements. DNS itself is a stumbling block for many people to self-hosting services, and it really wasn't meant to scale to billions of individual domains. There are hierarchical structers that could do this, including DNS hierarchies (domain, subdomain(s), host, user), or X-500 or LDAP, but which have not been widely adopted. Microsoft's Active Directory (an LDAP variant) exists, but is proprietary. I'm not sure what the solution here looks like, but it's actually among the key sticking points.
4. Support and privacy of metadata. The history of the standard RFC 822 message header structure is an interesting one, and dates to the early-20th century development of the inter-office memo. See Kathy Yates, "The Memorandum as Management Genre" (http://www.ismlab.usf.edu/dcom/Ch6_YatesMemoMgtCommQtly1989....). Among the surprises I found there, "In-reply-to" actually pre-dates" the "From", "To", and "Subject" fields -- creating chains of referenceable communications matters.* A key problem of current SMTP messaging is that these metadata are among the most useful to any surveillance system, capitalist or state, and yet are not protected by extant content-encryption systems. At the same time, some access to metadata may be critical for filtering purposes.
5. Various forms of trust and graduated trust-delegation. This applies at various levels. There's a level of trust which must generally be granted to message transport systems -- software, hardware, channels, protocols. Filtering against spam or unwanted email may involve some level of disclosure of either identity or contents. (It's likely that spam won't generally be (strongly) encrypted making filtering of it more viable, but this remains to be seen.) The ability to layer trust and encryption such that access, approval, and/or vouching may occur at various levels in a limited fashion could help.
(1/ cont)
The ability to restrict general messaging access to known trusted and vouched sources, to provide for bypass on an occasional basis, to allow third-party vetting (and to specify the trust/distrust level of same), etc., all strike me as necessary. Keep in mind that this applies to any messaging and comms channel, inclusive of postal mail and voice telecoms, both of which are also increasingly subject to nuisance attacks. I see the imminent death of conventional phone systems within a few years if the junk call problem isn't addressed, and most likely the solution to that being a disposal of number-dial-based systems as a resolution, also within a few years. Already, the abandonment of all phone-based comms by Millennials and many Gen-Xers is fairly widely reported.
7. The alternative, generally, to points 3-6 above, is the emergence of one or more exclusive messaging networks, following generally the selective and aspirational founder cohort as mentioned above. Note again that early comms systems tended to reflect this. Letter-writers, in the early 19th century, reflected the 5-25% of populations which were educated and literate. Telegraphs were limited in access to companies and wealthy individuals (historical note: president U.S. Grant learned of his electoral victory in the home of a neighbour in rural Galena, IL, who had a personal telegraph line installed). Telephones were initially uncommon, found in upscale households, a fact still pervasive enough in 1948 to lead to the famous "Dewey Beats Truman" Chicago Tribune headline based on the misleading sampling error of a teleophone-based survey. Long-distance and international dialing were limited through the 1950s and 1960s respectively. Long-distance charges were high through the 1990s (see above: marginal vs. fixed-rate pricing). Email was initially limited to a restricted set of college undergraduate and graduate students, faculty and staff, government departments, and technology-company professionals. The Blackberry was, for a time, a badge of membership amongst a professional elite. Each of these was a selective and aspirational cohort at its founding. The popularity of the media eventually, in virtually every case, killed the appeal of the addressible community, and did so most especially amongst the highly-valued members.
(One of the recurring themes of science fiction authors' essays of the 1960s - 1980s was the issue of postal mail, and the burdens this imposed on various authors. Arthur C. Clarke wrote on this several times, noting that eventually he'd be reduced to having to send only a pre-printed postcard, "Arthur C. Clarke regrets ...".
Attention is fundamentally limited. Rivality is the counterpoint to virality. The greater the reach of a medium, the more intrusive and annoying it is, axiomatically.
8. Standardisation of formats. I've long strongly favoured simple ASCII email. It's been ... amusing and validating to watch this issue be re-hashed with succeeding generations of messaging systems and devices. HTML allows far too many sins, ASCII not quite enough, but the balance seems to be toward simpler. I'd like to see a semantically structured message format emerge that has the concept of headers, emphasis, possibly even links, and perhaps images, but nothing more. I use the term "STML" (structured text markup language), though there's an extant concept of "POSH" (Plain-old Semantic HTML). The products of simple markup languages such as Markdown would generally work, though experience suggests that any markdown format is too much for the masses.
The ability of recipients to specify what messaging formats they are willing to accept, and have this occur as a negotiation at delivery time, strikes me as especially useful. "This recipient accepts only 7-bit ASCII unencoded messages", for example. Or lists of specifically whitelisted file formats, or encoding formats, by sender or sender category (family, friends, business, vendors, government, strangers, etc.). This serves to impose a complexity constraint on what a sender might attempt to deliver. Yes, you can try to send some highly-complex, highly-formatted newsletter, but 99.98% of recipients will reject it at delivery time, after all other checks have been completed.
Note that standardisation also might include the concept of forms-based email, something that's been ... exceedingly poorly supported to date. This is a concept addressed at length in the Kathy Yates article above about business correspondence standardisation. The ability to send an email that structures responses into a specific format (with, perhaps, a catch-all free-form field required) could be immensely useful.
Standardisation might also include calendar, lightweight messaging, voice/video, and other types of exchanges, all of which current messaging systems ... more or less completely fail at.
9. A re-thinking of authentication and validation. The ability to have the sender's identity be locally verified, and potentially cross-checked against multiple third-party systems could be highly useful. Presently, email quite literally allows anybody to claim to be anyone, simply by filling out the appropriate headers or text. A local identity validation would rely on local PKI validation, whist a third-party validation would send credentials to one or more third-party sites for confirmation. This would tremendously reduce the attack vector for misrepresentation, though of course it also produces risks of metadata leakage -- means for validating without divulging who is requesting the validation would help.
Creating one-off identities triggering validations might be an attack on this though -- the validation could only come from a specific recipient, or someone capable of accessign that party's messages. At the same time, one-off identities and their validation might be used as canaries indicating disclosure of private information, contacts lists, or other data.
Attacking the motives for phishing through improved authentication systems would of course also be useful.
I could continue, but let's end here.
(2/end)
Network bandwidth is not the problem.
The problem is disruption of effective communications channels.
I have three layers of anti-spam technology on my e-mail addresses, and still much of the traffic I get is spam.
If anyone would like to try give me an email address.
But if you are serious, I would like to know more what you are doing- filtering out all the emails is also one way to get 0 spam mails.