Re: [Orgmode] [bug] org-link-escape and (wrong-type-argument stringp nil)

[Sebastian Rose]

David Maus <address@hidden> writes:
> Sebastian Rose wrote:
>>David Maus <address@hidden> writes:
>>>>  sh$  man utf-8
>>>
>>> Thanks!  I finally get a grip on one of my personal nightmares.
>
>
>>It's not that bad, is it? :D
>
> Even better: It makes sense ;)
>
>>> The attached patch is the first step in this direction: It modifies
>>> the algorithm of `org-link-escape', now iterating over the input
>>> string with `mapconcat' and escaping all characters in the escape
>>> table or are between 127 and 255.
>
>>Between 128 (1000 0000) and 255 ??
>
>>The binary representation of 127 is 0111 1111 and valid ascii char. DEL
>>actually (sh$ man ascii)
>
> Right, and that's why it is encoded: No control characters in a URI.
>
> The final algorithm for the shiny new unicode aware percent encoding
> function would be:
>
>  - percent encode all characters in TABLE
>  - percent encode all characters below 32 and above 126
>    - encode the char in utf-8
>    - percent escape all bytes of the encoded char
>
> The remaining problem is keeping backward compatibility. There are Org
> files out there where "á" is encoded as "%E1" and not "%C3A1".  The
> percent decoding function should be able to recognize these old
> escapes and return the right value.


There is no chance to do it in a secure way.  But here's what's
possible.


These all work as expected:

(org-protocol-unhex-string "%E1")     ; á
(org-protocol-unhex-string "%A1")     ; ¡
(org-protocol-unhex-string "%E1%A1")  ; á¡
(org-protocol-unhex-string "%C3%B6")  ; still german ö


Also, capturing text from this page still works:
http://www.jnto.go.jp/jpn/


diff --git a/lisp/org-protocol.el b/lisp/org-protocol.el
index 21f28e7..f37ce1c 100644
--- a/lisp/org-protocol.el
+++ b/lisp/org-protocol.el
@@ -305,7 +305,7 @@ part."
 
 (defun org-protocol-unhex-string(str)
   "Unhex hexified unicode strings as returned from the JavaScript function
-encodeURIComponent. E.g. `%C3%B6' is the german Umlaut `ü'."
+encodeURIComponent. E.g. `%C3%B6' is the german Umlaut `ö'."
   (setq str (or str ""))
   (let ((tmp "")
        (case-fold-search t))
@@ -321,7 +321,11 @@ encodeURIComponent. E.g. `%C3%B6' is the german Umlaut 
`ü'."
 
 
 (defun org-protocol-unhex-compound (hex)
-  "Unhexify unicode hex-chars. E.g. `%C3%B6' is the German Umlaut `ü'."
+  "Unhexify unicode hex-chars. E.g. `%C3%B6' is the German Umlaut `ö'.
+Note: this function also decodes single byte encodings like
+`%E1' (\"á\") if not followed by another `%[A-F0-9]{2}' group.
+Singlebyte decoding is not secure though, since we could have
+two single byte characters above 128 in a row."
   (let* ((bytes (remove "" (split-string hex "%")))
         (ret "")
         (eat 0)
@@ -353,9 +357,22 @@ encodeURIComponent. E.g. `%C3%B6' is the german Umlaut 
`ü'."
        (setq val (logxor val xor))
        (setq sum (+ (lsh sum shift) val))
        (if (> eat 0) (setq eat (- eat 1)))
-       (when (= 0 eat)
+       (cond
+        ((= 0 eat)                         ;multi byte
          (setq ret (concat ret (org-protocol-char-to-string sum)))
          (setq sum 0))
+        ((not bytes)                       ; single byte(s)
+         (let ((bytes (remove "" (split-string hex "%")))
+               (ret ""))
+           (message "bytes: %s" bytes)
+
+           (while bytes
+             (let* ((b (pop bytes))
+                    (a (elt b 0))
+                    (b (elt b 1)))
+                    (setq ret
+                          (concat ret (char-to-string
+                                       (+ (lsh a 4) b)))))))))
        )) ;; end (while bytes
     ret ))
 


Best wishes

  Sebastian