help: cortex-a7 u-boot hang at pcie with gcc-linaro-4.9

[Liu Ting]

Hello guys,

I am newbie here and need your kindly help:)


When trying to use gcc-linaro-4.9 to build u-boot for ls1021atwr (ARM Cortex-A7 MPCore compliant with ARMv7-A architecture), we face issue. U-boot hangs at PCI-E.

After tracing the code, the issue is located at the line "*val = readl(addr);".



u-boot/drivers/pci/pcie_layerscape.c: ls_pcie_read_config():



if (PCI_BUS(d) == hose->first_busno) {

...

} else {

...

if (PCI_BUS(d) == hose->first_busno + 1) { #PCI_BUS(d) 1, hose->first_busno 0

ls_pcie_cfg0_set_busdev(pcie, busdev);

addr = pcie->va_cfg0 + (where & ~0x3); #pcie->va_cfg0 0x24000000, where 0xc

} else {

....

}

}

*val = readl(addr);

The gcc source we used is gcc-linaro-4.9-2015.02.tar.xz<https://releases.linaro.org/15.02/components/toolchain/gcc-linaro/4.9/gcc-linaro-4.9-2015.02.tar.xz> (link<https://releases.linaro.org/15.02/components/toolchain/gcc-linaro/4.9/>) which is based on FSF GCC 4.9.3-pre+svn220525.
Meanwhile, gcc-linaro-4.9-2015.01.tar.xz<https://releases.linaro.org/15.02/components/toolchain/gcc-linaro/4.9/gcc-linaro-4.9-2015.02.tar.xz> does not have this issue.
After Bisecting, we tracked down a gcc commit:
https://git.linaro.org/toolchain/gcc.git/commitdiff/e4f9e85e8152379aef373772b22075539920ffa2?hp=da278625cc33cbbc893cc50c2ac32fca31053ee8
2015-01-23 Jakub Jelinek <***@redhat.com>

PR rtl-optimization/63637
PR rtl-optimization/60663
* cse.c (merge_equiv_classes): Set new_elt->cost to MAX_COST
if elt->cost is MAX_COST for ASM_OPERANDS.
(find_sets_in_insn): Fix up comment typo.
(cse_insn): Don't set src_volatile for all non-volatile
ASM_OPERANDS in PARALLELs, but just those with multiple outputs
or with "memory" clobber. Set elt->cost to MAX_COST
for ASM_OPERANDS in PARALLEL. Set src_elt->cost to MAX_COST
if new_src is ASM_OPERANDS and elt->cost is MAX_COST.

* gcc.dg/pr63637-1.c: New test.
* gcc.dg/pr63637-2.c: New test.
* gcc.dg/pr63637-3.c: New test.
* gcc.dg/pr63637-4.c: New test.
* gcc.dg/pr63637-5.c: New test.
* gcc.dg/pr63637-6.c: New test.
* gcc.target/i386/pr63637-1.c: New test.
* gcc.target/i386/pr63637-2.c: New test.
* gcc.target/i386/pr63637-3.c: New test.
* gcc.target/i386/pr63637-4.c: New test.
* gcc.target/i386/pr63637-5.c: New test.
* gcc.target/i386/pr63637-6.c: New test.

git-svn-id: svn+ssh://gcc.gnu.org/svn/gcc/branches/gcc-4_9-***@220323 138bc75d<https://git.linaro.org/toolchain/gcc.git/object/138bc75d>-0d04-0410-961f-82ee72b054a4<https://git.linaro.org/toolchain/gcc.git/object/82ee72b054a4>

Before this commit, u-boot can boot up.

So any hint/suggestion? if more details needed, please feel free to tell us.

Thank you in advance.

-Ting

[Jim Wilson]

Post by Liu Ting
When trying to use gcc-linaro-4.9 to build u-boot for ls1021atwr (ARM
Cortex-A7 MPCore compliant with ARMv7-A architecture), we face issue. U-boot
hangs at PCI-E.
After tracing the code, the issue is located at the line “*val =
readl(addr);”.
https://git.linaro.org/toolchain/gcc.git/commitdiff/e4f9e85e8152379aef373772b22075539920ffa2?hp=da278625cc33cbbc893cc50c2ac32fca31053ee8
PR rtl-optimization/63637
PR rtl-optimization/60663
* cse.c (merge_equiv_classes): Set new_elt->cost to MAX_COST
...

It is hard to look at a compiler issue when I can't reproduce the
problem. You gave good info about the compiler, but you didn't point
at u-boot sources. i tried
git clone http://git.linaro.org/boot/u-boot-linaro-stable.git
but there is no pcie_layerscape.c file.

The problem fixed by the patch can be seen here
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63637
A description of the patch can be found in the submission email here:
https://gcc.gnu.org/ml/gcc-patches/2015-01/msg00894.html
There are testcases in the patch that show what it does.

The patch allows the compiler to optimize away redundant extended asm
statements via common subexpression elimination (CSE), when the asm
statements aren't volatile, don't have multiple outputs, and don't
have memory clobbers.

If this is breaking your code, then it appears that you are missing a
volatile declaration or extended asm memory clobber somewhere. I
don't have the code to look at, so I can't suggest what might be wrong
with the code.

Jim

[Liu Ting]

Hello Jim,

Thanks for your quick response. Some replies below. Thanks.

-Ting

-----Original Message-----
Sent: Thursday, September 03, 2015 7:14 AM
To: Liu Ting-B28495
B19537; Wang Huan-B18965
Subject: Re: help: cortex-a7 u-boot hang at pcie with gcc-linaro-4.9

Post by Liu Ting
When trying to use gcc-linaro-4.9 to build u-boot for ls1021atwr (ARM
Cortex-A7 MPCore compliant with ARMv7-A architecture), we face issue.
U-boot hangs at PCI-E.
After tracing the code, the issue is located at the line “*val =
readl(addr);”.
https://git.linaro.org/toolchain/gcc.git/commitdiff/e4f9e85e8152379aef
373772b22075539920ffa2?hp=da278625cc33cbbc893cc50c2ac32fca31053ee8
rtl-optimization/63637 PR rtl-optimization/60663
* cse.c (merge_equiv_classes): Set new_elt->cost to MAX_COST ...

It is hard to look at a compiler issue when I can't reproduce the problem.
You gave good info about the compiler, but you didn't point at u-boot sources.
i tried
git clone http://git.linaro.org/boot/u-boot-linaro-stable.git
but there is no pcie_layerscape.c file.

[Liu Ting-B28495] this is a file added for fsl ls1021atwr board. you can find it at:
http://git.freescale.com/git/cgit.cgi/ppc/sdk/u-boot.git/tree/drivers/pci/pcie_layerscape.c

The problem fixed by the patch can be seen here
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63637
https://gcc.gnu.org/ml/gcc-patches/2015-01/msg00894.html
There are testcases in the patch that show what it does.
The patch allows the compiler to optimize away redundant extended asm
statements via common subexpression elimination (CSE), when the asm statements
aren't volatile, don't have multiple outputs, and don't have memory clobbers.
If this is breaking your code, then it appears that you are missing a volatile
declaration or extended asm memory clobber somewhere. I don't have the code
to look at, so I can't suggest what might be wrong with the code.

[Liu Ting-B28495] thanks for your hint. It is very helpful.
BTW, we reverted below changes, then the issue disappeared:-)
- /* Also do not record result of a non-volatile inline asm with
- more than one result or with clobbers, we do not want CSE to
- break the inline asm apart. */
else if (GET_CODE (src) == ASM_OPERANDS
&& GET_CODE (x) == PARALLEL)
- sets[i].src_volatile = 1;
+ {
+ /* Do not record result of a non-volatile inline asm with
+ more than one result. */
+ if (n_sets > 1)
+ sets[i].src_volatile = 1;
+
+ int j, lim = XVECLEN (x, 0);
+ for (j = 0; j < lim; j++)
+ {
+ rtx y = XVECEXP (x, 0, j);
+ /* And do not record result of a non-volatile inline asm
+ with "memory" clobber. */
+ if (GET_CODE (y) == CLOBBER && MEM_P (XEXP (y, 0)))
+ {
+ sets[i].src_volatile = 1;
+ break;
+ }
+ }
+ }

Jim

[Huan Wang]

Hi, all,

Post by Liu Ting

Post by Jim Wilson

Post by Liu Ting
When trying to use gcc-linaro-4.9 to build u-boot for ls1021atwr (ARM
Cortex-A7 MPCore compliant with ARMv7-A architecture), we face

issue.

Post by Jim Wilson

Post by Liu Ting
U-boot hangs at PCI-E.
After tracing the code, the issue is located at the line “*val =
readl(addr);”.

https://git.linaro.org/toolchain/gcc.git/commitdiff/e4f9e85e8152379a

Post by Jim Wilson

Post by Liu Ting
ef
373772b22075539920ffa2?hp=da278625cc33cbbc893cc50c2ac32fca31053ee8
rtl-optimization/63637 PR rtl-optimization/60663
* cse.c (merge_equiv_classes): Set new_elt->cost to MAX_COST ...

It is hard to look at a compiler issue when I can't reproduce the

problem.

Post by Jim Wilson
You gave good info about the compiler, but you didn't point at u-boot

sources.

Post by Jim Wilson
i tried
git clone http://git.linaro.org/boot/u-boot-linaro-stable.git
but there is no pcie_layerscape.c file.

http://git.freescale.com/git/cgit.cgi/ppc/sdk/u-
boot.git/tree/drivers/pci/pcie_layerscape.c

Post by Jim Wilson
The problem fixed by the patch can be seen here
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63637
https://gcc.gnu.org/ml/gcc-patches/2015-01/msg00894.html
There are testcases in the patch that show what it does.
The patch allows the compiler to optimize away redundant extended asm
statements via common subexpression elimination (CSE), when the asm
statements aren't volatile, don't have multiple outputs, and don't

have memory clobbers.

Post by Jim Wilson
If this is breaking your code, then it appears that you are missing a
volatile declaration or extended asm memory clobber somewhere. I
don't have the code to look at, so I can't suggest what might be

wrong with the code.
[Liu Ting-B28495] thanks for your hint. It is very helpful.
BTW, we reverted below changes, then the issue disappeared:-)
- /* Also do not record result of a non-volatile inline asm with
- more than one result or with clobbers, we do not want CSE to
- break the inline asm apart. */
else if (GET_CODE (src) == ASM_OPERANDS
&& GET_CODE (x) == PARALLEL)
- sets[i].src_volatile = 1;
+ {
+ /* Do not record result of a non-volatile inline asm with
+ more than one result. */
+ if (n_sets > 1)
+ sets[i].src_volatile = 1;
+
+ int j, lim = XVECLEN (x, 0);
+ for (j = 0; j < lim; j++)
+ {
+ rtx y = XVECEXP (x, 0, j);
+ /* And do not record result of a non-volatile inline asm
+ with "memory" clobber. */
+ if (GET_CODE (y) == CLOBBER && MEM_P (XEXP (y, 0)))
+ {
+ sets[i].src_volatile = 1;
+ break;
+ }
+ }
+ }

[Alison Wang] After checking the code in u-boot, I found volatile is missing
when reading SCTLR register. As volatile is missing when reading SCTLR register
and SCTLR is set according to the value read from SCTLR, it causes CR_M bit is
not set. Then MMU is not enabled, the access to VA for PCIE fails.

I have sent a patch to upstream to fix this issue.
http://patchwork.ozlabs.org/patch/515666/

Thanks a lot for your support.


Best Regards,
Alison Wang