Cultural stereotypes retrieval from Google Autocomplete API

An American in Rome (Alberto Sordi)
A scene from the Italian movie “An American in Rome” (1954). Nando Mericoni is a young Italian living in the early ’50s in Rome. He loves the United States (that he discovers through the movies) and he tries to live like an American. The movie is a smart satire of “americanization”, plenty of stereotypes about Americans.

Stereotypes are often inconsistent generalizations based on cultural, social, sexual, linguistic, ethnic and religious differences.

Google offers an interesting autocomplete predictive algorithm that, while the user is typing, displays search queries based on millions of users’ search activities.
I developed an automatic web-based tool that allows the real-time extraction of cultural cliché and stereotypes based on Google autocomplete:

http://stereotypes.yellowjug.com

Free public APIs offered by Google and many other web services such as Facebook, Flickr and Twitter, can consitute a precious source of information since they allow to obtain extremely large and rich datasets that couldn’t be collected otherwise through standard means (e.g. questionnaires).

The cultural cliché and stereotypes retrieval tool aims to be a proof of concept of how data retrieved from the web can provide insights on human culture and beliefs (including but not limited to human ignorance and xenophobia). The results of our tool are retrieved in real time hence they cannot be previously moderated or controlled: take them with a grain of salt.

[EDIT January 2014: Google seems to have removed most of the stereotypes from the autocomplete features, therefore the Cultural stereotypes retrieval tool  is now presenting empty results in most of the cases]

Mailing List with Mailman and Phishing warning in Gmail

Gmail phishing alertGmail recently added a new anti-phishing feature that seems to not like too much mailing lists managed with Mailman (hosting providers often integrate this software in their control panel).

Several e-mails sent through those mailing lists are now “flagged” with the following warning:

This message may not have been sent by:

sendername@gmail.com Learn More Report Phishing

I currently have a couple of mailing list hosted by my personal server: due to the popularity of Gmail, this systematically displayed message is quite annoying, and tends to “alarm” some users of the mailing list.

If you are hosting a mailing list in your server, and you experience the same issue, here is a simple solution…

As Google recommends in the Gmail support section, you can create an SPF (Sender Policy Framework) record to your domain, so your mailing list will be authenticated.

To create an SPF record for your domain, access the Control Panel (e.g. CPanel) provided by your Hosting Company:

  1. Go to the section where you can edit the DNS (e.g. In Cpanel, “DNS -> Advanced DNS zone editor”)
  2. Add a new TXT record where the record name will be yourdomain.com. (NB. Do not add any www prefix, just the domain name and don’t forget the . after your domain name, it’s important!)
  3. In the TXT Data field, write: v=spf1 ip4:123.456.789.000 ~all (don’t forget to replace the IP in this example with your real IP)
  4. Leave the default value in the field TTL

(Needless to say/repeat, replace yourdomain.com with your domain name, and 123.456.789.000 with the IP of your server).

That’s it. There are different possible values you can add to your SPF record, they may be more or less “permissive” (e.g. emails sent just from some subdomains etc…). However the aforementioned example should work for everyone, if you need something more specific then start from here and follow the links.

It may take a while to have these changes to your DNS propagated worldwide (up to 48 hours), then, once the SPF record in your domain is active, the email sent through your mailing list(s) won’t display any more the phishing warning in Gmail. That’s it :)

How to check whether your SPF record works correctly?

  • Write your domain name in the field “Domain name:” (do not put any www., use just the domain name and the extension e.g. yourdomain.com) and press “Get SPF record (if any)“.
  • If the result is positive you’ll have a response which will look like “The TXT records found for your domain are:” etc… (the page will also display a validation of the record). Otherwise, you’ll obtain this error “No type SPF records found.” (This may mean that there is something wrong in your SPF record, or that your DNS are not propagated yet).

There is also a way to check whether your SPF record is active and works properly. You can enter Gmail and give a look to the headers of the emails sent through your mailing list.

The headers of the e-mails sent to a mailing list whose domain doesn’t have an SPF record, report a section similar to this one:

Received-SPF: neutral (google.com: 184.154.xxx.xxx is neither permitted nor denied by best guess record for domain of mailing-list@yourdomain.com) client-ip=184.154.xxx.xxx;

While an active SPF record in your domain would produce the following header:

Authentication-Results: mx.google.com; spf=pass (google.com: domain of mailing-list@yourdomain.com designates 184.154.xxx.xxx as permitted sender) smtp.mail=mailing-list@yourdomain.com; dkim=neutral (body hash did not verify) header.i=@gmail.com

Thus avoiding the annoying Phishing warning in the Gmail webmail interface.