[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
sms2samizdat-0.2 patch: open publishing of smses on newswire and/or cent
|
From: |
boud |
|
Subject: |
sms2samizdat-0.2 patch: open publishing of smses on newswire and/or centre column |
|
Date: |
Wed, 23 May 2007 02:36:03 +0200 (CEST) |
hi samizdat-devel,
This is a patch against 0.6.0.20070519-2.
There are several small changes from 0.1 to 0.2 of sms2samizdat.
FILES
(1) config.yaml
(2) .procmailrc
(3) frontpage_controller.rb
(4) resource_controller.rb
CHANGELOG:
A brief description of the changes:
* resource_controller should not rely on the prefix 'sms' for
sms2email files as defined in .procmailrc - in version 0.1 it was
obligatory that filename start with the string 'sms'.
** In 0.2 this is configurable in config.yaml with e.g.
filename_prefix=sms
as the default.
* CGI.escape:
Some sms companies escape non-alphanumeric characters e.g. ",
and although they should (i presume) add this header:
Content-Transfer-Encoding: quoted-printable
instead they only put these two headers:
Content-type: text/plain; charset=iso-8859-2
Content-transfer-encoding: 7BIT
** In 0.2 there's a parameter in config.yaml (by default commented
out) which if set, will call CGI.unescape( ) on both @subject and
@body of the sms/email:
unconvert_quoted_printable: true
It does require that Content-transfer-encoding: matches either 7BIT or
quoted-printable, which may reduce the number of cases in which it's
used unnecessarily.
However, i'm interested in what people think of the risks. Unescaping
anything is not a good idea unless you're reasonably sure that it should
be unescaped - otherwise you allow control characters etc. into your
system.
What do people think of the security aspect here?
* case insensitivity of email addresses (a mobile phone may by default
set the first letter of an address in upper case; in general case should
not matter).
** 0.2 - done
* requirement that username in From: matches /[0-9]{8}@/
** 0.2 - this requirement has been added so that it should block a fair
degree of (script kiddie type) people who might try to send ordinary
email (spam) to the address instead of sms'es. Other checks could be
possible, but in 0.1 there was no such constraint.
* protect against e.g. Subject: NaN, ensure that these are strings
** 0.2 @subject and @body are both .to_s'ed. Without this, sending
NaN as the Subject leads to an error because it leads to a nil title,
since it's considered to be a numerical value: nil.
OTHER COMMENTS
Installing this on a production site - on an old fedora distribution -
made it impossible (or not obvious anyway) how to use .procmailrc.
Sensitivity to directory/file privileges by both the MDA for incoming
email and by the apache user made it difficult for direct communication
between the two by the creation and reading/deletion of files. So
instead i wrote a simple hardwired script for cp-ing/chmoding/chowning,
which can be run by a cron job.
Should this script be part of the samizdat distribution? IMHO mail is
a non-trivial thing (because of the well-known risks of abuse and/or
human error and/or human ignorance of email standards) and creating a
general solution which works on a wide number of systems but is also
secure is not trivial. Procmail is fairly standard, so IMHO a
.procmailrc file should be enough as part of a "standard" version
of samizdat (if the version including an sms2samizdat interface looks
anything at all like my patch :).
The code fragment in resource_controller has got longer and longer,
more like typical fortran77 ;) than ruby oo code. Any should feel
welcome to modularise/oo-ify it more.
cheers
boud
On Thu, 17 May 2007, boud wrote:
(.) RISKS +
**COUNTERMEASURES
(1) email spam ** use spamassassin ** turn on only temporarily during rapid
events
** only publish the email address widely if enough people will be online
to openly edit them (or possibly moderate) - the sms's are published
with openediting turned on.
(2) SMS/MMSes are *removed* (for confidentiality) after publishing, even
if extraction of non-text content (photos etc) is not implemented
** In config.yaml turn on the parameter: do_not_delete_old_smses . A
sysadmin could then potentially publish the media items later.
...
PATCH AGAINST 0.6.0.20070519-2.
--- /tmp/tmp_snapshot/samizdat/lib/samizdat/controllers/resource_controller.rb
2007-05-02 20:43:56.000000000 +0200
+++ /usr/lib/ruby/1.8/samizdat/controllers/resource_controller.rb
2007-05-23 01:48:50.418271936 +0200
@@ -8,6 +8,8 @@
#
# vim: et sw=2 sts=2 ts=8 tw=0
+require 'mailread'
+
class ResourceController < Controller
def initialize(request, id=nil)
super
@@ -83,6 +203,140 @@
@layout = nil
end
+ def ResourceController.sms2message
+ if( config['sms2samizdat'] )
+ dir = config['sms2samizdat']['directory'].untaint
+ if (FileTest.directory?(dir) and FileTest.readable?(dir) and
+ FileTest.executable?(dir) and FileTest.writable?(dir))
+ prefix = config['sms2samizdat']['filename_prefix']
+ prefix = "" if !prefix
+ Dir.foreach(dir) do |filename|
+ # TODO: add more checks?
+ if( filename != "." and filename != ".." and
+ filename =~ Regexp.new('\A' + prefix) )
+ sms_mailfile = (dir + "/" + filename).untaint
+ if FileTest.readable?(sms_mailfile)
+ email= Mail.new(sms_mailfile)
+
+ @header = email.header
+ @body = email.body.join("\n")
+ @subject = @header['Subject']
+ @subject = "" if address@hidden # not all emails have subjects
+
+ # remove the subject if it may contain the sender's phone number
+ @subject = "" if @subject =~ /[0-9]{8}/
+
+ @from_number = ( @header['From'] =~ /[0-9]{8}@/ )
+
+ # this version is for smses only; mmses will be severely
truncated
+ min_body = config['sms2samizdat']['min_body'] or 30
+ max_body = config['sms2samizdat']['max_body'] or 200
+ if @body.length >= min_body
+ @body.squeeze!(" ")
+ @body = @body[0..(max_body-1)]
+ end
+
+ # MINIMAL SANITY CHECKS
+ # Is there enough content in the Subject: header and/or the body?
+ valid_message = false # conservative default
+ #
+ # copy subject/body to each other if one is much too short
+ if 0 == @subject.length and @body.length >= min_body
+ l = config['limit']['title'] or 20
+ @subject = @body.squeeze(" ")[0..(l-1)]
+ valid_message = true
+ elsif @body.length <= 1 and @subject.length >= min_body
+ @body = @subject
+ valid_message = true
+ end
+ if @subject.length >= min_body or @body.length >= min_body
+ valid_message = true
+ end
+
+ # protect against e.g. Subject: NaN, ensure that these are
strings
+ @subject = @subject.to_s
+ @body = @body.to_s
+
+ if config['sms2samizdat']['unconvert_quoted_printable'] and
+ @header['Content-transfer-encoding'] and
+ ( @header['Content-transfer-encoding'].downcase == "quoted-printable" or
+ @header['Content-transfer-encoding'].upcase == "7BIT" )
+ @subject = CGI.unescape(@subject)
+ @body = CGI.unescape(@body)
+ end
+ end
+
+ if valid_message and @from_number
+ @message = Message.new
+ @message.creator = Member.new(nil)
+ @member = @message.creator
+ @message.content = Content.new(nil, @member.login,
+ @subject, nil, @body)
+
+ # TODO: guess lang using email header, e.g. Content-Type:
+ @message.lang = config['locale']['languages'][0]
+
+ @message.desc = nil
+ @message.open = true # set_open
+
+ @message.insert!
+ @id = @message.id
+
+ # commit the transaction (db.commit doesn't seem to work)
+ db.transaction do |db|
+ end
+
+ if config['sms2samizdat']['event_email'] and
+ config['sms2samizdat']['event_focus_id'] and
+ config['sms2samizdat']['event_user_id'] and
+ @header['To']
+ if @header['To'] =~
Regexp.new(config['sms2samizdat']['event_email'],
+ Regexp::IGNORECASE)
+ #
+ # TODO this is mostly hacked from Focus::rating of
0.6.0.20070509
+ # and is too much "cut'n'paste'n'hack". Improve it!
+ #
+ rating =
+ Focus.validate_rating(config['sms2samizdat']['event_rating']) or 1
+ focus_id =
+ Resource.validate_id(config['sms2samizdat']['event_focus_id'])
+ member_id =
+ Resource.validate_id(config['sms2samizdat']['event_user_id'])
+ if focus_id and member_id
+ rdf.assert( %{
+UPDATE ?rating = :rating
+WHERE (rdf::subject ?stmt :related)
+ (rdf::predicate ?stmt dc::relation)
+ (rdf::object ?stmt #{focus_id})
+ (s::voteProposition ?vote ?stmt)
+ (s::voteMember ?vote :member)
+ (s::voteRating ?vote ?rating)},
+ { :rating => rating,
+ :related => @id, :member => member_id }
+ )
+
+ # commit the transaction
+ db.transaction do |db|
+ end
+
+ @rating = nil # invalidate rating cache #does this
function?
+ cache.flush
+ end # if focus_id and member_id
+ end # if @header['To'] =~ Regexp.new(config['event_email'])
+ end # if config['event_email'] and config['event_focus_id'] and
+
+ if FileTest.writable?(sms_mailfile)
+ File.delete(sms_mailfile) unless
+ config['sms2samizdat']['do_not_delete_old_smses']
+ end
+ end
+ end
+ end # Dir.foreach
+ end # if (FileTest.directory...
+ end # if (config('sms2samizdat'] )
+ end # def sms2message
+
+
private
# messages that are related to this focus
--- /tmp/tmp_snapshot/samizdat/lib/samizdat/controllers/frontpage_controller.rb
2007-05-07 22:14:58.000000000 +0200
+++ /usr/lib/ruby/1.8/samizdat/controllers/frontpage_controller.rb
2007-05-23 01:48:50.454266464 +0200
@@ -8,6 +8,8 @@
#
# vim: et sw=2 sts=2 ts=8 tw=0
+require 'samizdat/controllers/resource_controller'
+
class FrontpageController < Controller
def index
@@ -61,6 +65,8 @@
"</div>\n"
end
+ ResourceController.sms2message if( config['sms2samizdat'] )
+
if render_updates
updates_title = _('Recent Updates') +
(skip > 0? sprintf(_(', page %s'), skip + 1) : '')
--- /dev/null 2005-09-15 04:53:34.000000000 +0200
+++ /var/www/.procmailrc 2007-05-23 01:09:49.272180240 +0200
@@ -0,0 +1,50 @@
+######################################################################
+# sms2samizdat: .procmailrc
+#
+# Place this .procmailrc script in the home directory of the webserver
+# user (e.g. www-data) or add it the preexisting .procmailrc if it
+# exists already.
+
+# Directory for temporary holding of incoming sms-emails. This must be
+# accessible by the webserver user, so an error in the webserver
+# configuration could potentially make it accessible to remote users:
+# check your webserver configuration carefully.
+#
+# This directory must be entered in the samizdat yaml configuration
+# file for the site.
+SMS2SAMIZDAT_DIRECTORY=/var/www/imc-torun/tmp_sms/
+
+# Prefix for valid filenames - this may help protect against
+# accidentally publishing e.g. a README file that someone adds
+# to the directory.
+#
+# This directory must be entered in the samizdat yaml configuration
+# file for the site.
+SMS2SAMIZDAT_PREFIX=sms
+
+# Fields to remove from the email to protect confidentiality of
+# the sender. Do not rely on the default value suggested here -
+# check the resulting mail files for other identifying fields and
+# add them here using "-I ".
+#REMOVE_FIELDS="-I Return-Path: -I From: -I Message-id: -I Received:"
+REMOVE_FIELDS="-I Return-Path: -I Message-id: -I Received:"
+
+# Substitute 'sms2samizdat' here by the email user to whom remote users
+# send smses. It should be aliased to the webserver user,
+# e.g. www-data.
+:0:
+* ^To: address@hidden
+| formail ${REMOVE_FIELDS} -s sh -c \
+ 'sed -e "s/From .*/From address@hidden/g" > \
+ $SMS2SAMIZDAT_DIRECTORY/$SMS2SAMIZDAT_PREFIX`date +%Y%m%d%H%M%S`.$FILENO'
+
+# An additional email for smses to be linked to a focus. It should be
+# aliased to the webserver user, It must be listed in the config.yaml file
+# in order for the focus link to occur.
+:0:
+* ^To: address@hidden
+| formail ${REMOVE_FIELDS} -s sh -c \
+ 'sed -e "s/From .*/From address@hidden/g" > \
+ $SMS2SAMIZDAT_DIRECTORY/$SMS2SAMIZDAT_PREFIX`date +%Y%m%d%H%M%S`.$FILENO'
+
+######################################################################
--- /dev/null 2005-09-15 04:53:34.000000000 +0200
+++ /etc/samizdat/sites/config.yaml 2007-05-23 01:10:18.065802944 +0200
@@ -0,0 +1,41 @@
+# sms2samizdat patch 0.2 20070522 - highly experimental!
+#
+# 'directory' temporarily holds incoming sms-emails as individual
+# files. At intervals given by timeout: sms2samizdat: in defaults.yaml,
+# these files will be published as messages. The files are then removed
+# unless 'do_not_delete_old_smses' is set to true.
+# min_body: minimum number of characters required in the body of the message
+# max_body: maximum number of characters read from the body of the message
+#
+# Emails should be filtered to the directory using e.g. the .procmailrc example
+# in this patch.
+
+sms2samizdat:
+ directory: /var/www/samizdat/tmp_sms/
+# do_not_delete_old_smses: true
+ min_body: 30
+ max_body: 200
+#
+# (optional) prefix for filenames of emailed smses - other files are ignored
+ filename_prefix: sms
+#
+# Some sms2email gateways use "quoted printable" encoding of non-ascii
+# characters, e.g. %22 for the double quote symbol and fail to correctly
+# write this in email headers. If you wish to automatically unconvert
+# each such encoded character in the subject and body of smses back into
+# the original character, in cases where it is suspected that that might
+# have been used, then set unconvert_quoted_printable: true .
+# This could potentially be dangerous, so if you are unsure, comment out
+# this option.
+# unconvert_quoted_printable: true
+
+#
+# Emails to this _email_ will be linked to a _focus_ by a _user_ at a _rating_
+# as defined here. All of the first 3 parameters must be defined to enable this
+# feature. The rating parameter is optional (defaults to 1, i.e. will not go to
+# the centre column).
+ event_email: sms2event
+ event_focus_id: 2913
+ event_user_id: 3030
+ event_rating: 2
+
######################################################################