Crypto READ request failed (error=22). md10.eli[READ(offset=818688, length=512

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Crypto READ request failed (error=22). md10.eli[READ(offset=818688, length=512

Eitan Adler-4
vmcore.0 and kernel are available. I'll llkely spend some time this
weekend looking at the issues but if anyone cares to get to this
first:


https://reviews.freebsd.org/P166

----

Unread portion of the kernel message buffer:
[173352] GEOM_ELI: Device md10.eli created.
[173352] GEOM_ELI: Encryption: AES-XTS 128
[173352] GEOM_ELI:  Integrity: HMAC/SHA1
[173352] GEOM_ELI:     Crypto: hardware
[173352] GEOM_ELI: Crypto READ request failed (error=22).
md10.eli[READ(offset=818688, length=512
)]
[173352] panic: crypto_dispatch() failed (error=22)
[173352] cpuid = 20
[173352] time = 1523431733
[173352] KDB: stack backtrace:
[173352] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
0xfffffe00ae86d800
[173352] vpanic() at vpanic+0x18d/frame 0xfffffe00ae86d860
[173352] doadump() at doadump/frame 0xfffffe00ae86d8e0
[173352] g_eli_auth_run() at g_eli_auth_run+0x22b/frame 0xfffffe00ae86da00
[173352] g_eli_worker() at g_eli_worker+0x14c/frame 0xfffffe00ae86da70
[173352] fork_exit() at fork_exit+0x84/frame 0xfffffe00ae86dab0
[173352] fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ae86dab0
[173352] --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
[173352] KDB: enter: panic


#0  __curthread () at ./machine/pcpu.h:230
#1  doadump (textdump=0x1) at /usr/src/sys/kern/kern_shutdown.c:361
#2  0xffffffff80434f4c in db_fncall_generic (addr=<optimized out>,
rv=<optimized out>, nargs=<optimized out>, args=<optimized out>) at
/usr/src/sys/ddb/db_command.c:609
#3  db_fncall (dummy1=<optimized out>, dummy2=<optimized out>,
dummy3=<optimized out>, dummy4=<optimized out>) at
/usr/src/sys/ddb/db_command.c:657
#4  0xffffffff80434a99 in db_command (last_cmdp=<optimized out>,
cmd_table=<optimized out>, dopager=<optimized out>) at
/usr/src/sys/ddb/db_command.c:481
#5  0xffffffff80434814 in db_command_loop () at
/usr/src/sys/ddb/db_command.c:534
#6  0xffffffff80437a3f in db_trap (type=<optimized out>,
code=<optimized out>) at /usr/src/sys/ddb/db_main.c:250
#7  0xffffffff80babf53 in kdb_trap (type=0x3, code=0xffff0ff0,
tf=<optimized out>) at /usr/src/sys/kern/subr_kdb.c:697
#8  0xffffffff81024aa8 in trap (frame=0xfffffe00ae86d730) at
/usr/src/sys/amd64/amd64/trap.c:548
#9  <signal handler called>
#10 kdb_enter (why=0xffffffff8129f663 "panic", msg=<optimized out>) at
/usr/src/sys/kern/subr_kdb.c:479
#11 0xffffffff80b66b5a in vpanic (fmt=<optimized out>,
ap=0xfffffe00ae86d8a0) at /usr/src/sys/kern/kern_shutdown.c:826
#12 0xffffffff80b66920 in kassert_panic (fmt=0xffffffff825f243b
"crypto_dispatch() failed (error=%d)") at
/usr/src/sys/kern/kern_shutdown.c:723
#13 0xffffffff825ee10b in g_eli_auth_run (wr=0xfffff8003a580bc0,
bp=<optimized out>) at /usr/src/sys/geom/eli/g_eli_integrity.c:537
#14 0xffffffff825e9b7c in g_eli_worker (arg=<optimized out>) at
/usr/src/sys/geom/eli/g_eli.c:542
#15 0xffffffff80b26e34 in fork_exit (callout=0xffffffff825e9a30
<g_eli_worker>, arg=0xfffff8003a580bc0, frame=0xfffffe00ae86dac0) at
/usr/src/sys/kern/kern_fork.c:1039
#16 <signal handler called>

(kgdb) info registers
rax            0x12     0x12
rbx            0xfffffe00ae86d968       0xfffffe00ae86d968
rcx            0x80     0x80
rdx            0xfffffe00ae86d6f0       0xfffffe00ae86d6f0
rsi            0x80     0x80
rdi            0xffffffff81deab08       0xffffffff81deab08
rbp            0xfffffe00ae86da00       0xfffffe00ae86da00
rsp            0xfffffe00ae86d8f0       0xfffffe00ae86d8f0
r8             0x1      0x1
r9             0x0      0x0
r10            0xffffffff81cdc698       0xffffffff81cdc698
r11            0x0      0x0
r12            0x1e0    0x1e0
r13            0xfffff8004362f428       0xfffff8004362f428
r14            0xfffff8004362f598       0xfffff8004362f598
r15            0x18f800 0x18f800
rip            0xffffffff825ee10b       0xffffffff825ee10b <g_eli_auth_run+555>
eflags         0x82     [ SF ]
cs             0x20     0x20
ss             0x28     0x28
ds             <unavailable>
es             <unavailable>
fs             <unavailable>
gs             <unavailable>
fs_base        <unavailable>
gs_base        <unavailable>

(kgdb) info locals
sc = 0xfffff80136461c00
encr_secsize = 0x200
nsec = <optimized out>
lsec = 0x2
plaindata = 0xfffff800051b1a00
"\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336"...
data = <optimized out>
auth = 0xfffff8004362f414
"\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\067"
p = <optimized out>
i = <error reading variable i (Cannot access memory at address 0x1)>
decr_secsize = 0x200
dstoff = 0x18f800
error = 0x80
crp = 0xfffff8004362f428
data_secsize = <optimized out>
crde = <optimized out>
crda = <optimized out>
authkey = <optimized out>

(kgdb) info args
wr = 0xfffff8003a580bc0
bp = <optimized out>

(kgdb) p *crp
$1 = {
  crp_next = {
    tqe_next = 0xdeadc0dedeadc0de,
    tqe_prev = 0xdeadc0dedeadc0de
  },
  crp_task = {
    ta_link = {
      stqe_next = 0xdeadc0dedeadc0de
    },
    ta_pending = 0xc0de,
    ta_priority = 0xdead,
    ta_func = 0xdeadc0dedeadc0de,
    ta_context = 0xdeadc0dedeadc0de
  },
  crp_sid = 0x500000100000037,
  crp_ilen = 0x1f4,
  crp_olen = 0x1e0,
  crp_etype = 0x16,
  crp_flags = 0x60,
  {
    crp_buf = 0xfffff8004362f000 "",
    crp_mbuf = 0xfffff8004362f000,
    crp_uio = 0xfffff8004362f000
  },
  crp_opaque = 0xfffff803726ffc00,
  crp_desc = 0xfffff8004362f520,
  crp_callback = 0xffffffff825ee8f0 <g_eli_auth_read_done>,
  crp_tstamp = {
    sec = 0xdeadc0dedeadc0de,
    frac = 0xdeadc0dedeadc0de
  },
  crp_seq = 0xdeadc0de,
  crp_retw_id = 0x17
}

(kgdb) p *crp->crp_desc
$3 = {
  crd_skip = 0x14,
  crd_len = 0x1e0,
  crd_inject = 0x0,
  crd_flags = 0x10,
  CRD_INI = {
    cri_alg = 0x7,
    cri_klen = 0x100,
    cri_mlen = 0xdeadc0de,
    cri_key = 0xfffff8004362f598
"=\031eR\340\275D\366\205\020\307,\304]\bP\207\202\207\061\347\070\344\264\325\267\336\371~x\366\267\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336"...,
    cri_iv =
"\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336\336\300\255\336",
    cri_next = 0xdeadc0dedeadc0de
  },
  crd_next = 0xfffff8004362f4a8
}

(kgdb) p *crp->crp_desc->crd_next
$4 = {
  crd_skip = 0x14,
  crd_len = 0x1e0,
  crd_inject = 0xdeadc0de,
  crd_flags = 0x16,
  CRD_INI = {
    cri_alg = 0x16,
    cri_klen = 0x100,
    cri_mlen = 0xdeadc0de,
    cri_key = 0xfffff8003a28ae00
"\214\352\255\370\"\214|\275\367=\005\331X\325\251\371\220LN\207\037sY\305\274\033\346\200\217\343\272J\032H\021u[\024:\303\301\275\210\200\321\"\251\071\251\022",
    cri_iv =       "",
    cri_next = 0xdeadc0dedeadc0de
  },
  crd_next = 0x0
}

----


--
Eitan Adler

--
Eitan Adler
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Crypto READ request failed (error=22). md10.eli[READ(offset=818688, length=512

Allan Jude-9
On 04/11/2018 20:36, Eitan Adler wrote:

> vmcore.0 and kernel are available. I'll llkely spend some time this
> weekend looking at the issues but if anyone cares to get to this
> first:
>
>
> https://reviews.freebsd.org/P166
>
> ----
>
> Unread portion of the kernel message buffer:
> [173352] GEOM_ELI: Device md10.eli created.
> [173352] GEOM_ELI: Encryption: AES-XTS 128
> [173352] GEOM_ELI:  Integrity: HMAC/SHA1
> [173352] GEOM_ELI:     Crypto: hardware
> [173352] GEOM_ELI: Crypto READ request failed (error=22).
> md10.eli[READ(offset=818688, length=512
> )]

Do you happen to know how big the md(4) device is?

> [173352] panic: crypto_dispatch() failed (error=22)
> [173352] cpuid = 20
> [173352] time = 1523431733
> [173352] KDB: stack backtrace:
> [173352] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xfffffe00ae86d800
> [173352] vpanic() at vpanic+0x18d/frame 0xfffffe00ae86d860
> [173352] doadump() at doadump/frame 0xfffffe00ae86d8e0
> [173352] g_eli_auth_run() at g_eli_auth_run+0x22b/frame 0xfffffe00ae86da00
> [173352] g_eli_worker() at g_eli_worker+0x14c/frame 0xfffffe00ae86da70
> [173352] fork_exit() at fork_exit+0x84/frame 0xfffffe00ae86dab0
> [173352] fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00ae86dab0
> [173352] --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
> [173352] KDB: enter: panic
>
>
> #0  __curthread () at ./machine/pcpu.h:230
> #1  doadump (textdump=0x1) at /usr/src/sys/kern/kern_shutdown.c:361
> #2  0xffffffff80434f4c in db_fncall_generic (addr=<optimized out>,
> rv=<optimized out>, nargs=<optimized out>, args=<optimized out>) at
> /usr/src/sys/ddb/db_command.c:609
> #3  db_fncall (dummy1=<optimized out>, dummy2=<optimized out>,
> dummy3=<optimized out>, dummy4=<optimized out>) at
> /usr/src/sys/ddb/db_command.c:657
> #4  0xffffffff80434a99 in db_command (last_cmdp=<optimized out>,
> cmd_table=<optimized out>, dopager=<optimized out>) at
> /usr/src/sys/ddb/db_command.c:481
> #5  0xffffffff80434814 in db_command_loop () at
> /usr/src/sys/ddb/db_command.c:534
> #6  0xffffffff80437a3f in db_trap (type=<optimized out>,
> code=<optimized out>) at /usr/src/sys/ddb/db_main.c:250
> #7  0xffffffff80babf53 in kdb_trap (type=0x3, code=0xffff0ff0,
> tf=<optimized out>) at /usr/src/sys/kern/subr_kdb.c:697
> #8  0xffffffff81024aa8 in trap (frame=0xfffffe00ae86d730) at
> /usr/src/sys/amd64/amd64/trap.c:548
> #9  <signal handler called>
> #10 kdb_enter (why=0xffffffff8129f663 "panic", msg=<optimized out>) at
> /usr/src/sys/kern/subr_kdb.c:479
> #11 0xffffffff80b66b5a in vpanic (fmt=<optimized out>,
> ap=0xfffffe00ae86d8a0) at /usr/src/sys/kern/kern_shutdown.c:826
> #12 0xffffffff80b66920 in kassert_panic (fmt=0xffffffff825f243b
> "crypto_dispatch() failed (error=%d)") at
> /usr/src/sys/kern/kern_shutdown.c:723
> #13 0xffffffff825ee10b in g_eli_auth_run (wr=0xfffff8003a580bc0,
> bp=<optimized out>) at /usr/src/sys/geom/eli/g_eli_integrity.c:537
> #14 0xffffffff825e9b7c in g_eli_worker (arg=<optimized out>) at
> /usr/src/sys/geom/eli/g_eli.c:542
> #15 0xffffffff80b26e34 in fork_exit (callout=0xffffffff825e9a30
> <g_eli_worker>, arg=0xfffff8003a580bc0, frame=0xfffffe00ae86dac0) at
> /usr/src/sys/kern/kern_fork.c:1039
> #16 <signal handler called>
>


--
Allan Jude
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: Crypto READ request failed (error=22). md10.eli[READ(offset=818688, length=512

Eitan Adler-4
On 11 April 2018 at 18:27, Allan Jude <[hidden email]> wrote:

> On 04/11/2018 20:36, Eitan Adler wrote:
>> vmcore.0 and kernel are available. I'll llkely spend some time this
>> weekend looking at the issues but if anyone cares to get to this
>> first:
>>
>>
>> https://reviews.freebsd.org/P166
>>
>> ----
>>
>> Unread portion of the kernel message buffer:
>> [173352] GEOM_ELI: Device md10.eli created.
>> [173352] GEOM_ELI: Encryption: AES-XTS 128
>> [173352] GEOM_ELI:  Integrity: HMAC/SHA1
>> [173352] GEOM_ELI:     Crypto: hardware
>> [173352] GEOM_ELI: Crypto READ request failed (error=22).
>> md10.eli[READ(offset=818688, length=512
>> )]

∴sudo kyua --loglevel=debug test sys/geom/class/eli/init_test

is how to reproduce

and it fails right after

[13174 04:59:40.967 eax@fasteagle /usr/tests !2!]∴sudo kyua
--loglevel=debug test sys/geom/class/eli/init_test
sys/geom/class/eli/init_test:init  ->  failed: Miscompare for
ealgo=aes-xts keylen=128 sec=512  [0.070s]
sys/geom/class/eli/init_test:init_B  ->  passed  [3.507s]
sys/geom/class/eli/init_test:init_J  ->  passed  [78.789s]
sys/geom/class/eli/init_test:init_a  ->



--
Eitan Adler
_______________________________________________
[hidden email] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[hidden email]"