Re: PSPP-BUG: PSPP Version 079 - SPS with German Characters-Possible Bug

[ajk-eis]

OK, I tried the console here too.  Using the console the file is read in
correctly and can be exported (correctly) into a portable file (por).

I can successfully read the por file into the UI (meaning without errors
being written into the output window), HOWEVER all of the strings (variable
labels and values) are cut off at the first occurrence of a national
character!  So actually I have a lot of import errors that are not being
logged or shown in the UI output.

This means "ausgewählt" becomes "ausgew". "ueberflüssig" becomes "ueberfl".
And so on.

So this really seems to be the UI that can't handle the national characters.
Weird.  I hope and assume that you are seeing the actual characters there.
They are not showing up on the web site but instead are replaced by "?"
(ASCII instead of UTF-8)

I've included the por file that I created from the console.  Trouble is, my
users can and will not use the console ....  This is not an "emergency".  We
will be going live around July.

Alle

> -----Ursprüngliche Nachricht-----
> Von: John Darrington [mailto:address@hidden
> Gesendet: Samstag, 5. Mai 2012 23:41
> An: ajk-eis
> Cc: 'John Darrington'; address@hidden; 'jeff'
> Betreff: Re: PSPP-BUG: PSPP Version 079 - SPS with German Characters-
> Possible Bug
> 
> It seems that there is indeed a bug.
> 
> We'll look into it.
> 
> J'
> 
> On Sat, May 05, 2012 at 08:59:56PM +0200, ajk-eis wrote:
>      Thanks for bearing with us on the weekend!
> 
>      >> that?).  I had to move the lines BEGIN DATA and all following
> directly
>      >> after DATA LIST.  However this shouldn't have caused problems
> reading ...
>      Moving that section behind DATA LIST did not really make a big
> difference
>      other than allowing the Data to be read in before the errors starting
>      occurring. (See attachments) Also the "Bad character U+FFFD" does not
> show
>      up at the moment.  However we have no problem reading files in with
> "our"
>      ordering, as long as there are no national characters.
> 
>      >> certainly shouldn't cause a crash.
>      The last stable release 2012-03-15 does not crash.  The RCs 03-20 and
> 04-20
>      did (on running the old file with the data at the bottom AND national
>      characters.
> 
>      >> Anyway I suggest you make this correction and see how you go.
>      I just changed our generator to move the data up to the top. A new
> sps file
>      is included with a new Output PDF. All I use is the GUI in Win 7 (no
>      console). Since the data is at the top it is read in correctly (there
> are no
>      national characters before it anymore). I still get many errors and
> in fact
>      the displayed text in the Output Window is NOT the same as in the sps
> file
>      (look at the line with the variable label q470c475 - it should read
> as the
>      line above "Wie finden ...." - but it doesn't). This is static text.
> The
>      beginning of each of these labels q470c472 through q470c476
> originates with
>      the exact same string.  The problem occurs due to the word "h??lich"
> in line
>      q470c474 which has two national characters.
> 
>      The problem continues at the line "value labels q470c475" which
> apparently
>      actually reads "val u e l abel s q470c475".  This to me is amazing
> (can't
>      see it in the PDF but copy it out into something else) but again is
> probably
>      due to the national characters in the previous lines?  In fact the
> "Nf" from
>      the variable label shows up again ("valNf").
> 
>      Alle
> 
>      > -----Urspr?ngliche Nachricht-----
>      > Von: John Darrington [mailto:address@hidden
>      > Gesendet: Samstag, 5. Mai 2012 19:15
>      > An: ajk-eis
>      > Cc: address@hidden
>      > Betreff: Re: PSPP-BUG: PSPP Version 079 - SPS with German
> Characters -
>      > Possible Bug
>      >
>      > Thanks for sending these files, and thanks for the bug report.
>      >
>      > I'm not seeing the problems that you are seeing, although not
>      > everything is quite right.
>      >
>      > There is a small problem with your syntax:  You have the VARIABLE
> LABELS,
>      > MISSING VALUES and VALUE LABELS commands situated between the DATA
> LIST
>      > and
>      > BEGIN DATA commands.  I don't think this is allowed in PSPP (does
> SPSS
>      > allow
>      > that?).  I had to move the lines BEGIN DATA and all following
> directly
>      > after
>      > DATA LIST.  However this shouldn't have caused problems reading the
> file,
>      > and
>      > certainly shouldn't cause a crash.
>      >
>      > When I make this change, everything works fine using the terminal,
>      > although
>      > there is a problem with the GUI.
>      >
>      > Anyway I suggest you make this correction and see how you go.
>      >
>      > J'
>      >
>      >
>      > On Sat, May 05, 2012 at 05:43:29PM +0200, ajk-eis wrote:
>      >      Hello John, thanks for the quick reply and moving this to
> bugs.
>      >
>      >      Attached are 4 files.
>      >      1 - umfrage-7-Errors.sps - This file is UTF-8 encoded and
> contains
>      > German national characters (if this e-mail text is Unicode - ??,
> ??, ??).
>      >      (U+00E4, U+00F6, U+00FC, U+00DF)
>      >      These are not all the possibilities but are enough to test
> with. This
>      > file will not run here and produces consistent import errors. Not
>      > contained but very popular in Europe is also the Euro Sign (??? -
> U+20AC)
>      > which will also create the errors.
>      >
>      >      2 - PSPPIRE Output Viewer_7_Errors.pdf - This is the output of
> PSPP
>      > with the file above (with national characters) which is not
> imported.
>      >
>      >      3 - umfrage-7a-OK.sps - This file is exactly the same
> generated file
>      > but we have replaced the occurrences of the 4 characters above with
> the
>      > "denglish" equivalents (i.e. ae, oe, ue, ss). IOW there are only
> ASCII
>      > characters contained. This file will run without errors.
>      >
>      >      4 - PSPPIRE Output Viewer_7a_OK.pdf - This is the output of
> PSPP with
>      > the file in 3 above, without errors and successfully imported.
>      >
>      >      I hope that you can reproduce the problem there with file 1.
>      >
>      >      Alle
>      >
>      >      > -----Urspr??ngliche Nachricht-----
>      >      > Von: John Darrington [mailto:address@hidden
>      >      > Gesendet: Samstag, 5. Mai 2012 14:58
>      >      > An: ajk-eis
>      >      > Cc: address@hidden; 'jeff'
>      >      > Betreff: Re: PSPP Version 079 - SPS with German Characters -
>      > Possible Bug
>      >      >
>      >      > [Moving the thread to address@hidden
>      >      >
>      >      > I wasn't aware of this.
>      >      >
>      >      > Please do send examples.
>      >      >
>      >      > J'
>      >      >
>      >      > On Sat, May 05, 2012 at 12:51:11PM +0200, ajk-eis wrote:
>      >      >      Installation - PSPP build pspp-079-20120315-32bits
>      >      >      OS - Win 7 Pro (up to date) and UBUNTU
>      >      >
>      >      >      I assume you already know this but in case you don't we
> are
>      > not able
>      >      > to import / load any sps file that contains any special
> German
>      > characters
>      >      > in the values. The output is "bad character U+FFFD". The
> same
>      > occurs with
>      >      > the Euro Symbol ("???"). This occurs on a Windows 7
> installation
>      > and a
>      >      > LINUX installation. These are valid UTF8 files.  This would
> appear
>      > to be a
>      >      > regression since the import of the same files works with
> 0.7.8.xxx.
>      >      >
>      >      >      We have tried creating the sps file with all imaginable
>      > encodings
>      >      > (i.e. UTF 8, UTF 16, CP 1252 (which is standard for local
>      > installation),
>      >      > etc.) with no resolution of the problem. At the moment we
> must use
>      > ASCII
>      >      > encoding which has no national characters or replace all
> special
>      > German
>      >      > characters prior to the import.
>      >      >
>      >      >      The RC 2012-03-20 and 2012-04-11, installed on Windows
> 7,
>      > crash when
>      >      > the import file is run.
>      >      >
>      >      >      In the PSPP Version pspp-master-20111111 (0.7.8.xxx)
> the
>      > import works
>      >      > without problems but this version is not stable on a
> (German)
>      > Windows 7
>      >      > installation.
>      >      >
>      >      >      I can supply example files if desired.
>      >      >
>      >      >      Cheers
>      >      >      Alle
>      >      >
>      >      >
>      >      >      _______________________________________________
>      >      >      Pspp-users mailing list
>      >      >      address@hidden
>      >      >      https://lists.gnu.org/mailman/listinfo/pspp-users
>      >      >
>      >      > --
>      >      > PGP Public key ID: 1024D/2DE827B3
>      >      > fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8
> 27B3
>      >      > See http://keys.gnupg.net or any PGP keyserver for public
> key.
>      >
>      >
>      >
>      >
>      >
>      >
>      >      _______________________________________________
>      >      Bug-gnu-pspp mailing list
>      >      address@hidden
>      >      https://lists.gnu.org/mailman/listinfo/bug-gnu-pspp
>      >
>      >
>      > --
>      > PGP Public key ID: 1024D/2DE827B3
>      > fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
>      > See http://keys.gnupg.net or any PGP keyserver for public key.
> 
> 
> 
> 
> 
> --
> PGP Public key ID: 1024D/2DE827B3
> fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
> See http://keys.gnupg.net or any PGP keyserver for public key.

export.por
Description: Binary data