syndication feed (pull feeds) hack

[boud]

hi samizdat people,

On Wed, 13 Sep 2006, Antono Vasiljev wrote:

> I've just installed samizdat from experimental. I need some time to look
> at it more closely and chose relevant task :) Podcasting feature is
> interesting for me so maybe i'll try to implement it.
>
> And... Some monthes ago i've trying to implement items exchange based on
> samizdat. But now i tottaly forgot the things :) I still have interest
> to this topic.

Cool :)

antono - nice to see you here :).

My update to 0.5.5.20060914-1*deb from experimental went smoothly - i
just had to copy/split my old config.yaml file into

/etc/samizdat/default.yaml
/etc/samizdat/sites.yaml
/etc/samizdat/site/[SITENAME].yaml

and the server seems to be running normally on my local machine.


dmitry - i definitely think having the config parameters in 
/etc/samizdat/ is an improvement over the previous version. i like
the idea of mir, which is that all "customisations" can be placed 
in etc/ (except that for mir, the idea is to compile everything in 
place in the /var/www/<sitename>/ hierarchy, which to me sounds rather
unreasonable - in mir, the etc/ is within this /var/www/... hierarchy,
not in /etc/ ) - maybe things like site versions of css files and so on
could also go in /etc/samizdat/ ? or else in /var/www/... ?  Anyway, 
this is probably not (yet) a priority issue.



Below i've put my first hack solution to syndication feeds as a patch
on  0.5.5.20060914-1.

i'm sure people can do much better than this - it's hardwired with two
example feeds, it is probably not very robust against errors, it
probably should be broken off into a separate module, it doesn't (yet)
use date information (neither in display nor to sort the items from
different feeds together as one superfeed), and i haven't written
any documentation about it either.

But it does seem to work :) and if a ruby newbie like me can do this,
then it at least proves that samizdat is relatively easy to add stuff
to.

Below are two minor comments at the usability/documentation level, and
then the hack patch for the syndication feed. (i will try to do
something better after feedback from the list.)

cheers
boud



MINOR COMMENTS

* /etc/samizdat/defaults.yaml:

** email - Warn the person installing that s/he may comment 
out address: and sendmail: if s/he doesn't want to or can't send email
from the host server.

** timeout:
i suggest putting in more reasonable defaults - remember that mediawiki
timeout is about 10-15 minutes and we expect many people to use this CMS
from internet cafes, libraries, etc.  - personalised preferences could
later on able users to modify these.

#  login: 7776000   # 60 * 60 * 24 * 90 = 90 days
#  last: 2592000   # 60 * 60 * 24 * 30 = 30 days
#  moderate: 86400   # 60 * 60 * 24 = 1 day
#  cache: 300   # 5 * 60 = 5 minutes
  login: 3600   # 60 * 60  = 1 hour
  last: 1800   # 60 * 30  = 0.5 hours
  moderate: 1800   # 60 * 30  = 0.5 hours
  cache: 300   # 5 * 60 = 5 minutes


SYNDICATION PATCH

The two files with the extension  _0.5.5.20060914  are 
the originals from Thursday 14 Sep's debian experimental version.


----------------------------------------------------------------------


--- /usr/lib/cgi-bin/samizdat/index.rb_0.5.5.20060914   2006-09-14 
14:36:39.000000000 +0200
+++ /usr/lib/cgi-bin/samizdat/index.rb  2006-09-17 05:28:36.738803064 +0200
@@ -172,15 +172,69 @@
       t.nav_rss(rss_updates) + t.nav(updates.size, skip + 1))
   end

+# HACK by boud 17.09.2006 
+  require 'net/http'
+  require 'rss/1.0'
+  require 'rss/dublincore'
+  require 'rss/2.0'
+
+  import_feeds_title = _('Syndication Feeds')
+
+  rss_pre= "<li><a href=\""
+  rss_post= "\">"
+  rss_final= "</a></li><br />\n"
+
+# HARDWIRED - should come from e.g. /etc/samizdat/
+  import_feeds_list = [
+    ['pl.indymedia.org', '/pl/syndycation/plindy-features.rdf'],
+    ['cia.bzzz.net', '/node/feed']
+    ]
+
+  if render_features  # HARDWIRED to render_features
+    import_feeds_body = "<ul>"
+
+    for feed_number in 0...import_feeds_list.length
+      feed_host = Net::HTTP.new(import_feeds_list[feed_number][0], 80)
+      response, data = feed_host.get(import_feeds_list[feed_number][1], nil)
+
+      begin 
+        rss = RSS::Parser.parse(response.body)
+      rescue RSS::InvalidRSSError
+        rss = RSS::Parser.parse(response.body, false)
+      end
+ 
+      rss_channel = rss
+      if rss.rss_version == "2.0"
+        rss_channel = rss.channel
+      end
+
+      for item_number in 0...rss_channel.items.length
+        if rss_channel.item(item_number).do_validate
+          rss_link= rss_channel.item(item_number).
+            link.sub(/[ \n\t]*(http[^ \n\t]*)[ \n\t]*/,'\1')
+          import_feeds_body =  import_feeds_body +
+            rss_pre + rss_link + rss_post + 
+            rss_channel.item(item_number).title + rss_final
+       end
+      end
+    end # for feed_number in ...
+    import_feeds_body = import_feeds_body + "</ul>"
+  end   # if render_features
+
+
+
   page =
     if full_front_page
 %{<table>
   <thead>
-    
<tr><th>#{focuses_title}</th><th>#{features_title}</th><th>#{updates_title}</th></tr>
+    <tr><th>#{focuses_title}</th><th>#{features_title}</th>
+    <th>#{import_feeds_title}</th>
+    <th>#{updates_title}</th></tr>
   </thead>
   <tr>
     <td class="focuses">#{focuses}</td>
     <td class="features" rowspan="3">#{features}</td>
+    <td class="import_feeds" rowspan="3">#{import_feeds_body}</td>
     <td class="updates" rowspan="3">#{updates}</td>
   </tr>
   <tr><td class="links-head">}+_('Links')+'</td></tr>


----------------------------------------------------------------------


--- /usr/share/samizdat/css/indy.css_0.5.5.20060914     2006-04-17 
15:17:35.000000000 +0200
+++ /usr/share/samizdat/css/indy.css    2006-09-17 02:49:40.120589000 +0200
@@ -134,7 +134,8 @@
 td.focuses { width: 15%; height: 100%; text-align: left; }
 td.links-head { height: 100%; font-weight: bold; background: #555; }
 td.links { text-align: left; font-size: small; }
-td.features { width: 55%; text-align: left; }
+td.features { width: 40%; text-align: left; }
+td.import_feeds { width: 15%; text-align: left; }
 td.updates { background: #036; }
 td.updates ul li.even { background: #024; }


----------------------------------------------------------------------