RE: [Freeipmi-devel] FW: [bug #24300] ipmiconsole cannot connect to various Intel servers

[Al Chu]

Hey Andrew,

Hmmm, the BMC Busy error seems justified.

telco64: ------------------
telco64: [              93h] = message_tag[ 8b]
telco64: [               1h] = rmcpplus_status_code[ 8b]

1h = "Insufficient resources to create a session"

So I think this error is legit given the response.  Any idea why the
board would respond with this type of error code?  Could it be to many
failed connection attempts?  I remember an Intel IPMI 1.5 board I played
with would not be very responsive if there were many failed connections
in a row.

telco64: =====================================================
telco64: IPMI 2.0 Open Session Request
telco64: =====================================================
<snip>
telco64: IPMI Command Data:
telco64: ------------------
telco64: [              C5h] = message_tag[ 8b]
telco64: [               0h] = requested_maximum_privilege_level[ 4b]

telco64: =====================================================
telco64: IPMI 2.0 Open Session Response
telco64: =====================================================
<snip>
telco64: IPMI Command Data:
telco64: ------------------
telco64: [              C5h] = message_tag[ 8b]
telco64: [               0h] = rmcpplus_status_code[ 8b]
telco64: [               0h] = maximum_privilege_level[ 4b]

(ipmiconsole_checks.c,
ipmiconsole_check_open_session_response_privilege, 610):
hostname=telco64; protocol_state=0x2: open session response privilege
check failed; p = 3^M
[error received]: privilege level cannot be obtained for this user

I'll have to look into this.  0h = IPMI_PRIVILEGE_LEVEL_HIGHEST_LEVEL,
which should not be done for the Intel boards when the intel workaround
is specified.

      /* IPMI Workaround
       *
       * Intel IPMI 2.0 implementations don't support the highest level
privilege.
       */
      if (c->config.workaround_flags &
IPMICONSOLE_WORKAROUND_INTEL_2_0_SESSION)
        privilege_level = c->config.privilege_level;
      else
        privilege_level = IPMI_PRIVILEGE_LEVEL_HIGHEST_LEVEL;

Al

On Mon, 2008-12-08 at 06:38 -0800, Cress, Andrew R wrote:
> Al,
> 
> Attached are the debug logs of the two errors I get.
> Same errors for any Intel server, the "BMC Busy" error in fi.log is the most 
> common.
> 
> Andy
>  
> 
> -----Original Message-----
> From: Al Chu [mailto:address@hidden 
> Sent: Friday, December 05, 2008 5:41 PM
> To: Cress, Andrew R
> Subject: RE: [Freeipmi-devel] FW: [bug #24300] ipmiconsole cannot connect to 
> various Intel servers
> 
> Hey Andrew,
> 
> If you have time, do you think you could sanity check this tar.gz's
> ipmiconsole and some of the other stuff (ipmipower and ipmi-sensors
> would hit all the ipmi 2.0 "implementations") against an Intel machine.
> I'd appreciate it.
> 
> http:// ftp.zresearch.com/pub/freeipmi/qa-
> release/freeipmi-0.7.4.beta0.tar.gz 
> 
> Al
> 
> On Fri, 2008-12-05 at 11:31 -0800, Cress, Andrew R wrote:
> > Yep, I meant to save the draft and go finish that part but hit send 
> > instead.  :-)
> > 
> > Andy 
> > 
> > -----Original Message-----
> > From: Al Chu [mailto:address@hidden 
> > Sent: Friday, December 05, 2008 2:29 PM
> > To: Cress, Andrew R
> > Cc: address@hidden
> > Subject: RE: [Freeipmi-devel] FW: [bug #24300] ipmiconsole cannot connect 
> > to various Intel servers
> > 
> > Hey Andrew,
> > 
> > On Fri, 2008-12-05 at 10:54 -0800, Cress, Andrew R wrote:
> > > Al,
> > > 
> > >       /* IPMI Workaround (achu)
> > >        *
> > >        * Discovered on SE7520AF2 with Intel Server Management Module
> > >        * (Professional Edition)
> > >        *
> > >        * The username must be padded despite explicitly not being
> > >        * allowed.  "No Null characters (00h) are allowed in the name".
> > >        * Table 13-11 in the IPMI 2.0 spec.
> > >        */
> > > Hmmm.  This is because when the set username command is issued, the
> > > command data is padded with nulls to 16 bytes, as shown in secion
> > > 22.28, and that is common across most of the spec.  But Table 13-11 is
> > > an exception to the rest of IPMI username handling, in that it is not
> > > a fixed-length 16-byte entity.  This is implemented the same in most
> > > Intel BMCs that are currently available.  In ipmiutil, this RAKP1
> > > stuff is handled the same way that ipmitool does, by ...
> > 
> > Forgot to write the end to this part? :-)
> > 
> > Looking through the ipmiutil code, it doesn't seem like any padding is
> > done.  I wrote the Intel 2.0 workarounds a long time ago and probably
> > added things to "just make them work", not seeing if there was a cleaner
> > way to deal with it.
> > 
> > Al
> > 
> > > 
> > >       /* IPMI Workaround (achu)
> > >        *
> > >        * Discovered on SE7520AF2 with Intel Server Management Module
> > >        * (Professional Edition)
> > >        *
> > >        * When the authentication algorithm is HMAC-MD5-128 and the
> > >        * password is greater than 16 bytes, the Intel BMC truncates the
> > >        * password to 16 bytes when generating keys, hashes, etc.  So we
> > >        * have to do the same when generating keys, hashes, etc.
> > >        */
> > > I doubt that the IMM firmware supported 20-byte passwords (the first 
> > > Intel BMC with IPMI 2.0), but some other Intel BMCs do, I believe.  
> > > Ipmiutil currently does not handle 20-byte passwords, and that could be 
> > > done conditionally on certain systems, but if we are talking about 
> > > managing passwords for hundreds of servers where some support 16 bytes 
> > > and some support 20 bytes, all of the passwords would be restricted to 
> > > 16-bytes anyway, and the IPMI 2.0 spec requires support/compatibility for 
> > > 16-byte passwords for this reason.  We don't have any customers who have 
> > > a use-case where 20-byte passwords would be used, so I haven't 
> > > implemented the logic for it.  
> > > 
> > > 
> > > -----Original Message-----
> > > From: address@hidden [mailto:address@hidden On Behalf Of Al Chu
> > > Sent: Friday, December 05, 2008 1:27 PM
> > > To: Cress, Andrew R
> > > Cc: address@hidden
> > > Subject: Re: [Freeipmi-devel] FW: [bug #24300] ipmiconsole cannot connect 
> > > to various Intel servers
> > > 
> > > Hey Andrew,
> > > 
> > > On Fri, 2008-12-05 at 09:14 -0800, Cress, Andrew R wrote:
> > > > Retry, first send failed.
> > > > 
> > > > -----Original Message-----
> > > > From: Cress, Andrew R [mailto:address@hidden 
> > > > Sent: Friday, December 05, 2008 8:43 AM
> > > > To: Albert Chu; Bryan Henderson; Andy Cress; address@hidden
> > > > Subject: RE: [bug #24300] ipmiconsole cannot connect to various Intel 
> > > > servers
> > > > 
> > > > Al,
> > > > 
> > > > This was the first time I had tried ipmiconsole, so I don't know if it 
> > > > worked before or what changed.  
> > > > 
> > > > For an example of what is different with Intel boards, you can view the 
> > > > source to ipmitool or ipmiutil under the 'lanplus' protocol.  It boils 
> > > > down to some different assumptions about defaults or special conditions.
> > > > In ipmitool, the syntax requires specifying "-o intelplus", but 
> > > > ipmiutil detects the manufacturer/product id first and doesn't need 
> > > > those options.  
> > > > 
> > > > From ipmiutil:
> > > > lib/lanplus/lanplus.c:is_sol_partial_ack() has an intelplus special 
> > > > case, which probably should apply to all other boards too (?)
> > > 
> > > Yeah, I'm not quite sure why this would be an intel corner case.  Does
> > > intel return 0 for the accepted_character_count even if it accepted all
> > > the data?  So a return of 0 is just assumed to be equivalent to "all
> > > data accepted"??  I don't currently handle this case in ipmiconsole.
> > > The assumption is if you didn't accept any data, then you resend data
> > > just as if there was a partial acceptance.
> > > 
> > > > lib/lanplus/lanplus.c:ipmi_lanplus_open_session() has an intelplus 
> > > > condition for privilege defaults
> > > 
> > > I handle this one:
> > > 
> > >       /* IPMI Workaround
> > >        *
> > >        * Intel IPMI 2.0 implementations don't support the highest level 
> > > privilege.
> > >        */
> > >       if (c->config.workaround_flags & 
> > > IPMICONSOLE_WORKAROUND_INTEL_2_0_SESSION)
> > >         privilege_level = c->config.privilege_level;
> > >       else
> > >         privilege_level = IPMI_PRIVILEGE_LEVEL_HIGHEST_LEVEL;
> > > 
> > > > lib/lanplus/lanplus_crypt.c:lanplus_rakp4_hmac_matches() has two 
> > > > intelplus cases
> > > 
> > > handle this one:
> > > 
> > >   /* IPMI Workaround
> > >    *
> > >    * Intel IPMI 2.0 implementations respond with the integrity check
> > >    * value based on the integrity algorithm rather than the
> > >    * authentication algorithm.
> > >    */
> > >   if (c->config.workaround_flags & 
> > > IPMICONSOLE_WORKAROUND_INTEL_2_0_SESSION)
> > >     {
> > >       if (c->config.integrity_algorithm == IPMI_INTEGRITY_ALGORITHM_NONE)
> > >         authentication_algorithm = 
> > > IPMI_AUTHENTICATION_ALGORITHM_RAKP_NONE;
> > >       else if (c->config.integrity_algorithm == 
> > > IPMI_INTEGRITY_ALGORITHM_HMAC_SHA1_96)
> > >         authentication_algorithm = 
> > > IPMI_AUTHENTICATION_ALGORITHM_RAKP_HMAC_SHA1;
> > >       else if (c->config.integrity_algorithm == 
> > > IPMI_INTEGRITY_ALGORITHM_HMAC_MD5_128)
> > >         authentication_algorithm = 
> > > IPMI_AUTHENTICATION_ALGORITHM_RAKP_HMAC_MD5;
> > >       else if (c->config.integrity_algorithm == 
> > > IPMI_INTEGRITY_ALGORITHM_MD5_128)
> > >         /* achu: I have not been able to reverse engineer this.  So 
> > > accept it */
> > >         return 1;
> > >     }
> > >   else
> > >     authentication_algorithm = c->config.authentication_algorithm;
> > > 
> > > > lib/lanplus/lanplus_crypt.c:lanplus_generate_rakp3_authcode() has an 
> > > > intelplus case for privilege defaults
> > > 
> > > Ahh, this might be it.  In ipmipower and libfreeipmi's ipmi 2.0 code I
> > > handle this properly.  But in ipmiconsole I seem to have accidentally
> > > put this code in the RAKP1 section.  That might be the reason that I'm
> > > checking the return value from the RAKP2 incorrectly.  
> > > 
> > > I'll do some more auditing to see if I can find why the other fellow's
> > > example isn't working on the INTELs.  I noticed he is using a NULL
> > > username/password.  You mind given my next tar.gz a test run to see if I
> > > caught everything?
> > > 
> > > BTW, it seems as though ipmiutil does not implement all of the Intel
> > > workarounds I found.  There were a number of corner cases for
> > > username/password lengths.  Here's what I have in my comments.
> > > 
> > >       /* IPMI Workaround (achu)
> > >        *
> > >        * Discovered on SE7520AF2 with Intel Server Management Module
> > >        * (Professional Edition)
> > >        *
> > >        * The username must be padded despite explicitly not being
> > >        * allowed.  "No Null characters (00h) are allowed in the name".
> > >        * Table 13-11 in the IPMI 2.0 spec.
> > >        */
> > > 
> > >       /* IPMI Workaround (achu)
> > >        *
> > >        * Discovered on SE7520AF2 with Intel Server Management Module
> > >        * (Professional Edition)
> > >        *
> > >        * When the authentication algorithm is HMAC-MD5-128 and the
> > >        * password is greater than 16 bytes, the Intel BMC truncates the
> > >        * password to 16 bytes when generating keys, hashes, etc.  So we
> > >        * have to do the same when generating keys, hashes, etc.
> > >        */
> > > 
> > > Does ipmiutil handle these too?  Or is it possible Intel fixed some
> > > issues but not others in newer firmware?
> > > 
> > > Al
> > > 
> > > > That's all that is different.
> > > > 
> > > > Andy
> > > > 
> > > > -----Original Message-----
> > > > From: Albert Chu [mailto:address@hidden 
> > > > Sent: Thursday, December 04, 2008 5:47 PM
> > > > To: Albert Chu; Bryan Henderson; Andy Cress; address@hidden
> > > > Subject: [bug #24300] ipmiconsole cannot connect to various Intel 
> > > > servers
> > > > 
> > > > 
> > > > Follow-up Comment #2, bug #24300 (project freeipmi):
> > > > 
> > > > Sorry I didn't see these posts earlier.  Hopefully I've fixed the 
> > > > config on
> > > > Savannah so that bugs actually send out e-mails to the mailing list.
> > > > 
> > > > I implemented the Intel workarounds a long time ago, but no longer have 
> > > > an
> > > > Intel motherboard.  So I've been forward porting the patches since then 
> > > > and
> > > > praying they still work and I didn't mess anything up along the way.  I 
> > > > guess
> > > > something is messed up or there is something new to workaround.
> > > > 
> > > > Hopefully I can find an Intel mobo to try and fix this on.  I'm going 
> > > > through
> > > > the code right now visually and can't see a workaround issue.
> > > > 
> > > > Al
> > > > 
> > > > P.S.  Bryan, I can see how the wording of the manpage was 
> > > > misinterpreted to
> > > > make you think "I note the manual mentions this can happen with
> > > > --workaround=intel20, but it doesn't mention anything to do about it. 
> > > > ".  I'm
> > > > going to fix up the manpage to instead say:
> > > > 
> > > > There are a number of Intel IPMI 2.0 bugs.  These problems may cause
> > > > "username invalid", "password invalid", or "k_g invalid" errors to 
> > > > occur. 
> > > > They can be worked around by specifying the "intel20" workaround.
> > > > 
> > > > 
> > > >     _______________________________________________________
> > > > 
> > > > Reply to this item at:
> > > > 
> > > >   <http://    savannah.gnu.org/bugs/?24300>
> > > > 
> > > > _______________________________________________
> > > >   Message sent via/by Savannah
> > > >   http://    savannah.gnu.org/
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > _______________________________________________
> > > > Freeipmi-devel mailing list
> > > > address@hidden
> > > > http://    lists.gnu.org/mailman/listinfo/freeipmi-devel
> > > > 
-- 
Albert Chu
address@hidden
Computer Scientist
High Performance Systems Division
Lawrence Livermore National Laboratory