[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
sms2samizdat-0.1 patch: open publishing of smses on newswire and/or cent
|
From: |
boud |
|
Subject: |
sms2samizdat-0.1 patch: open publishing of smses on newswire and/or centre column |
|
Date: |
Thu, 17 May 2007 02:32:48 +0200 (CEST) |
hi samizdat-devel
Here's version 0.1 of sms2samizdat. If there are no unfixed bugs
reported within the next week, then it is likely to be tested in a
real (big) event, but on just one small local independent media site.
The patch below is against 0.6.0.20070506. i'll probably test against
20070509 soon and i'll post an updated patch if the present patch fails
against it.
FILES
(1) defaults.yaml + config.yaml
(2) .procmailrc
(3) frontpage_controller.rb
(4) resource_controller.rb
(.) RISKS +
**COUNTERMEASURES
(1) email spam
** use spamassassin
** turn on only temporarily during rapid events
** only publish the email address widely if enough people will be online
to openly edit them (or possibly moderate) - the sms's are published
with openediting turned on.
(2) SMS/MMSes are *removed* (for confidentiality) after publishing, even
if extraction of non-text content (photos etc) is not implemented
** In config.yaml turn on the parameter: do_not_delete_old_smses . A
sysadmin could then potentially publish the media items later.
TODO
(1) publishing is delayed by caching mechanisms
** either users must frequently publish articles/coments or the caching
delay should be set short enough to frequently show updates
** calling a samizdat ruby script by .procmailrc would solve this problem,
but potentially allow blocking the website through an email avalanche.
(2) MMS (multimedia messaging service)
** http://rubyforge.org/projects/mms2r/ 1.0.7 April 27, 2007
- strips advertising from MMSes, specific parameters of a bunch of
US-based(?) mobile phone companies are in ruby files and .yml files
(3) Lots of checks missing on sysadmin errors, e.g. if non-mail files
e.g. binary are copied to/created in the directory where the sms emails
should lie.
** (easy) Add some more sanity checks before taking the file seriously.
COMMENTS
The special event option enables linking smses sent to a defined email
address automatically to the focus for that event. This requires
using either an existing user or (better) creating a special "focus
user" by hand, as in a standard gnu/linux system which has a whole
bunch of nonhuman special users ;), e.g. www-data, postgres, ...
In principle this means that the people participating in an action
can present a live, centre column feed without requiring any moderator
(nor sysadmin, unless there's a bug) intervention.
The only delay in publishing is due to caching - either a user's
intervention (adding a reply, editing an article, publishing an
article) is required or the cache delay needs to pass. Decreasing the
cache timeout during a big event would increase the rapidity of the
updates, but also slow down the number of pages served per
second. However, given that users will notice the obvious update to
the content, they may consider this desirable.
The un-caching process for event smses is a two step process, since it
occurs through frontpage_controller.rb. This is not necesarily a bad
thing. It gives users with logins some chance to improve the article,
remove advertising signatures, or moderators get a chance to hide
spam articles before they get to the central column - but all these
are *post*-publication, not pre-publication events - in the
indymedia/wiki/open-publishing tradition.
cheers
boud
--- /tmp/tmp_snapshot/samizdat/lib/samizdat/controllers/resource_controller.rb
2007-05-02 20:43:56.000000000 +0200
+++ /usr/lib/ruby/1.8/samizdat/controllers/resource_controller.rb
2007-05-17 01:00:25.735575344 +0200
@@ -8,6 +8,8 @@
#
# vim: et sw=2 sts=2 ts=8 tw=0
+require 'mailread'
+
class ResourceController < Controller
def initialize(request, id=nil)
super
@@ -83,6 +203,118 @@
@layout = nil
end
+ def ResourceController.sms2message
+ if( config['sms2samizdat'] )
+ dir = config['sms2samizdat']['directory'].untaint
+ if (FileTest.directory?(dir) and FileTest.readable?(dir) and
+ FileTest.executable?(dir) and FileTest.writable?(dir))
+ Dir.foreach(dir) do |filename|
+ if(filename != "." and filename != ".." and
+ "sms" == filename[0..2] ) # TODO: add more checks?
+ sms_mailfile = (dir + "/" + filename).untaint
+ email= Mail.new(sms_mailfile)
+ @header = email.header
+ @body = email.body.join("\n")
+ @subject = @header['Subject']
+ @subject = "" if address@hidden # not all emails have subjects
+
+ # remove the subject if it may contain the sender's phone number
+ @subject = "" if @subject =~ /[0-9]{8}/
+
+ # this version is for smses only; mmses will be severely truncated
+ min_body = config['sms2samizdat']['min_body'] or 30
+ max_body = config['sms2samizdat']['max_body'] or 200
+ if @body.length >= min_body
+ @body.squeeze!(" ")
+ @body = @body[0..(max_body-1)]
+ end
+
+ # MINIMAL SANITY CHECKS
+ # Is there enough content in the Subject: header and/or the body?
+ valid_message = false # conservative default
+ #
+ # copy subject/body to each other if one is much too short
+ if 0 == @subject.length and @body.length >= min_body
+ l = config['limit']['title'] or 20
+ @subject = @body.squeeze(" ")[0..(l-1)]
+ valid_message = true
+ elsif @body.length <= 1 and @subject.length >= min_body
+ @body = @subject
+ valid_message = true
+ end
+ if @subject.length >= min_body or @body.length >= min_body
+ valid_message = true
+ end
+
+
+ if valid_message
+ @message = Message.new
+ @message.creator = Member.new(nil)
+ @member = @message.creator
+ @message.content = Content.new(nil, @member.login,
+ @subject, nil, @body)
+
+ # TODO: guess lang using email header, e.g. Content-Type:
+ @message.lang = config['locale']['languages'][0]
+
+ @message.desc = nil
+ @message.open = true # set_open
+
+ @message.insert!
+ @id = @message.id
+
+ # commit the transaction (db.commit doesn't seem to work)
+ db.transaction do |db|
+ end
+
+ if config['sms2samizdat']['event_email'] and
+ config['sms2samizdat']['event_focus_id'] and
+ config['sms2samizdat']['event_user_id'] and
+ @header['To']
+ if @header['To'] =~
Regexp.new(config['sms2samizdat']['event_email'])
+ #
+ # TODO this is mostly hacked from Focus::rating of
0.6.0.20070509
+ # and is too much "cut'n'paste'n'hack". Improve it!
+ #
+ rating =
+ Focus.validate_rating(config['sms2samizdat']['event_rating']) or 1
+ focus_id =
+ Resource.validate_id(config['sms2samizdat']['event_focus_id'])
+ member_id =
+ Resource.validate_id(config['sms2samizdat']['event_user_id'])
+ if focus_id and member_id
+ rdf.assert( %{
+UPDATE ?rating = :rating
+WHERE (rdf::subject ?stmt :related)
+ (rdf::predicate ?stmt dc::relation)
+ (rdf::object ?stmt #{focus_id})
+ (s::voteProposition ?vote ?stmt)
+ (s::voteMember ?vote :member)
+ (s::voteRating ?vote ?rating)},
+ { :rating => rating,
+ :related => @id, :member => member_id }
+ )
+
+ # commit the transaction
+ db.transaction do |db|
+ end
+
+ @rating = nil # invalidate rating cache #does this
function?
+ cache.flush
+ end # if focus_id and member_id
+ end # if @header['To'] =~ Regexp.new(config['event_email'])
+ end # if config['event_email'] and config['event_focus_id'] and
+
+ File.delete(sms_mailfile) unless
+ config['sms2samizdat']['do_not_delete_old_smses']
+ end
+ end
+ end # Dir.foreach
+ end # if (FileTest.directory...
+ end # if (config('sms2samizdat'] )
+ end # def sms2message
+
+
private
# messages that are related to this focus
--- /tmp/tmp_snapshot/samizdat/lib/samizdat/controllers/frontpage_controller.rb
2007-05-05 14:15:07.000000000 +0200
+++ /usr/lib/ruby/1.8/samizdat/controllers/frontpage_controller.rb
2007-05-17 01:43:42.762767200 +0200
@@ -8,6 +8,8 @@
#
# vim: et sw=2 sts=2 ts=8 tw=0
+require 'samizdat/controllers/resource_controller'
+
class FrontpageController < Controller
def index
@@ -61,6 +65,8 @@
"</div>\n"
end
+ ResourceController.sms2message if( config['sms2samizdat'] )
+
if render_updates
updates_title = _('Recent Updates') +
(skip > 0? sprintf(_(', page %s'), skip + 1) : '')
--- /dev/null 2005-09-15 04:53:34.000000000 +0200
+++ /var/www/.procmailrc 2007-05-16 23:11:44.475957496 +0200
@@ -0,0 +1,39 @@
+######################################################################
+# sms2samizdat: .procmailrc
+#
+# Place this .procmailrc script in the home directory of the webserver
+# user (e.g. www-data) or add it the preexisting .procmailrc if it
+# exists already.
+
+# Directory for temporary holding of incoming sms-emails. This must be
+# accessible by the webserver user, so an error in the webserver
+# configuration could potentially make it accessible to remote users:
+# check your webserver configuration carefully.
+#
+# This directory must be entered in the samizdat yaml configuration
+# file for the site.
+SMS2SAMIZDAT_DIRECTORY=/var/www/imc-torun/tmp_sms/
+
+# Fields to remove from the email to protect confidentiality of
+# the sender. Do not rely on the default value suggested here -
+# check the resulting mail files for other identifying fields and
+# add them here using "-I ".
+REMOVE_FIELDS="-I Return-Path: -I From: -I Message-id: -I Received:"
+
+# Substitute 'sms2samizdat' here by the email user to whom remote users
+# send smses. It should be aliased to the webserver user,
+# e.g. www-data.
+:0:
+* ^To: address@hidden
+| formail ${REMOVE_FIELDS} -s sh -c \
+ 'sed -e "s/From .*/From address@hidden/g" > $SMS2SAMIZDAT_DIRECTORY/sms`date +%Y%m%d%H%M%S`.$FILENO'
+
+# An additional email for smses to be linked to a focus. It should be
+# aliased to the webserver user, It must be listed in the config.yaml file
+# in order for the focus link to occur.
+:0:
+* ^To: address@hidden
+| formail ${REMOVE_FIELDS} -s sh -c \
+ 'sed -e "s/From .*/From address@hidden/g" > $SMS2SAMIZDAT_DIRECTORY/sms`date +%Y%m%d%H%M%S`.$FILENO'
+
+######################################################################
--- /old/etc/samizdat/sites/config.yaml 2005-09-15 04:53:34.000000000 +0200
+++ /etc/samizdat/sites/config.yaml 2007-05-17 01:41:39.206550600 +0200
@@ -0,0 +1,27 @@
+# sms2samizdat patch 0.1 20070517 - highly experimental!
+#
+# 'directory' temporarily holds incoming sms-emails as individual
+# files. At intervals given by timeout: sms2samizdat: in defaults.yaml,
+# these files will be published as messages. The files are then removed
+# unless 'do_not_delete_old_smses' is set to true.
+# min_body: minimum number of characters required in the body of the message
+# max_body: maximum number of characters read from the body of the message
+#
+# Emails should be filtered to the directory using e.g. the .procmailrc example
+# in this patch.
+
+sms2samizdat:
+ directory: /var/www/imc-torun/tmp_sms/
+# do_not_delete_old_smses: true
+ min_body: 30
+ max_body: 200
+#
+# Emails to this _email_ will be linked to a _focus_ by a _user_ at a _rating_
+# as defined here. All of the first 3 parameters must be defined to enable this
+# feature. The rating parameter is optional (defaults to 1, i.e. will not go to
+# the centre column).
+# event_email: sms2event
+# event_focus_id: 2913
+# event_user_id: 3030
+# event_rating: 2
+
- sms2samizdat-0.1 patch: open publishing of smses on newswire and/or centre column,
boud <=