Patch for ABE on Power Macintosh.

[Jens Bauer]

Hi all.

I've incorrectly reported a build failure in the support system.
<https://support.linaro.org/hc/en-us/requests/1876>
-But I'm posting a copy to this list, in order to follow Victor's advice.

I've suggested a couple of changes, which makes ABE a little more compatible.
Though I can still not build the toolchain on my PowerBook G4, I think that my two modifications might ease building on Intel based Macs and perhaps other architectures in addition.

In short:
* Use getconf instead of /proc/cpuinfo (you're already using getconf in configure.ac).
* By default use twice as many cores for -jN as usual. (If you have 2 cores, use -j4)
* Change the architecture from 'Power Macintosh' (which contains an offending space) to 'powerpc'.

...And here's "configure.ac.patch":

---8<-----8<-----8<-----
diff --git a/configure.ac b/configure.ac
index 1bf3593..8cc4986 100644
--- a/configure.ac
+++ b/configure.ac
@@ -49,9 +49,10 @@ AC_SUBST(DBPASSWD)
DBHOST=${dbhost}
AC_SUBST(DBHOST)

-CPUS="`grep -c proces /proc/cpuinfo`"
+
+let CPUS=2*`getconf _NPROCESSORS_ONLN`
AC_SUBST(CPUS)
-CORES="`grep cores /proc/cpuinfo | tail -1 | cut -d ' ' -f 3`"
+#CORES="`grep cores /proc/cpuinfo | tail -1 | cut -d ' ' -f 3`"
AC_SUBST(CORES)
LIBC="`getconf GNU_LIBC_VERSION`"
AC_SUBST(LIBC)
@@ -59,6 +60,7 @@ KERNEL="`uname -r`"
AC_SUBST(KERNEL)
BUILDHOST="`${srcdir}/config.guess`"
BUILD_ARCH="`uname -m`"
+[[ "`uname -s`" == "Darwin" ]] && BUILD_ARCH="`uname -p`"
AC_SUBST(BUILD_ARCH)
AC_SUBST(BUILDHOST)
HOSTNAME="`uname -n`"
--->8----->8----->8-----

Apart from that, I'd like to report a successful build on Cubieboard2 running Armbian.
I built for the aarch64 architecture, and I'm currently attempting to build Armbian for beelink GT1 using the new shiny Linaro toolchain. :)
...and what a pleasure, there's no fan-noise when building on the Cubieboard2!


Love
Jens

[Christophe Lyon]

Post by Jens Bauer
Hi all.

Hi Jens,

Post by Jens Bauer
I've incorrectly reported a build failure in the support system.
<https://support.linaro.org/hc/en-us/requests/1876>
-But I'm posting a copy to this list, in order to follow Victor's advice.
I've suggested a couple of changes, which makes ABE a little more compatible.

Thanks for your interest and efforts.

Post by Jens Bauer
Though I can still not build the toolchain on my PowerBook G4, I think that my two modifications might ease building on Intel based Macs and perhaps other architectures in addition.
* Use getconf instead of /proc/cpuinfo (you're already using getconf in configure.ac).
* By default use twice as many cores for -jN as usual. (If you have 2 cores, use -j4)
* Change the architecture from 'Power Macintosh' (which contains an offending space) to 'powerpc'.
---8<-----8<-----8<-----
diff --git a/configure.ac b/configure.ac
index 1bf3593..8cc4986 100644
--- a/configure.ac
+++ b/configure.ac
@@ -49,9 +49,10 @@ AC_SUBST(DBPASSWD)
DBHOST=${dbhost}
AC_SUBST(DBHOST)
-CPUS="`grep -c proces /proc/cpuinfo`"
+
+let CPUS=2*`getconf _NPROCESSORS_ONLN`

Using getconf is indeed more portable, I think.
However, I'm not sure we want the '2*' part of the patch,
since we sort-of rely on the -j factor in our validations
(e.g. number of independent builds in parallel on a
given machine)

Post by Jens Bauer
AC_SUBST(CPUS)
-CORES="`grep cores /proc/cpuinfo | tail -1 | cut -d ' ' -f 3`"
+#CORES="`grep cores /proc/cpuinfo | tail -1 | cut -d ' ' -f 3`"
AC_SUBST(CORES)

It looks like CORES isn't used anywhere, so it's simpler to
just remove these lines.

Post by Jens Bauer
LIBC="`getconf GNU_LIBC_VERSION`"
AC_SUBST(LIBC)
@@ -59,6 +60,7 @@ KERNEL="`uname -r`"
AC_SUBST(KERNEL)
BUILDHOST="`${srcdir}/config.guess`"
BUILD_ARCH="`uname -m`"
+[[ "`uname -s`" == "Darwin" ]] && BUILD_ARCH="`uname -p`"

I have no way of testing this, I'll check with someone else.

Thanks for the suggestions.

Post by Jens Bauer
AC_SUBST(BUILD_ARCH)
AC_SUBST(BUILDHOST)
HOSTNAME="`uname -n`"
--->8----->8----->8-----
Apart from that, I'd like to report a successful build on Cubieboard2 running Armbian.
I built for the aarch64 architecture, and I'm currently attempting to build Armbian for beelink GT1 using the new shiny Linaro toolchain. :)
...and what a pleasure, there's no fan-noise when building on the Cubieboard2!

:-)

Christophe

Post by Jens Bauer
Love
Jens
_______________________________________________
linaro-toolchain mailing list
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

[Jens Bauer]

Hi Christophe.

Thanks for the review; that was quick. :)

Post by Christophe Lyon
Thanks for your interest and efforts.

You deserve some contributions/freebies, with all the hard work you're already doing - thank you on behalf of thousands of people using your toolchain!

Post by Christophe Lyon

Post by Jens Bauer
* By default use twice as many cores for -jN as usual. (If you have 2 cores, use -j4)
-CPUS="`grep -c proces /proc/cpuinfo`"
+
+let CPUS=2*`getconf _NPROCESSORS_ONLN`

Using getconf is indeed more portable, I think.
However, I'm not sure we want the '2*' part of the patch,
since we sort-of rely on the -j factor in our validations
(e.g. number of independent builds in parallel on a
given machine)

Fair enough. :)
-If you use the CPUS variable for the actual number of cores, then you could do like I do in my builds instead:

let CPUS=`getconf _NPROCESSORS_ONLN`
let cpucount=2*$CPUS
pmake="make -j$cpucount"; [ -w "$prefix" ] && smake="$pmake" || smake="sudo $pmake"
case `uname -s` in 'Darwin') isMac=1; smake="sudo $pmake" ;; 'Linux') isLinux=1 ;; esac

-I mentioned this in the support request as well. I'm creating two variables, which contains the code used for executing make. It has a 'built-in' -jN, where N is twice the number of cores.

-But if you're somehow querying make for the -j argument, then you'll probably need to use $CPUS directly.
(Normally -j2 will use around 50% CPU-time on a dual core machine - this is most likely due to the disk access involved, where the CPU waits for the disk to deliver data and sleeps until the data arrives).


Love
Jens

[Christophe Lyon]

Post by Jens Bauer
Hi Christophe.
Thanks for the review; that was quick. :)

Post by Christophe Lyon
Thanks for your interest and efforts.

You deserve some contributions/freebies, with all the hard work you're already doing - thank you on behalf of thousands of people using your toolchain!

You may have noticed that your suggestions have been integrated:
- support for Darwin
- use of getconf

Sorry, I realize that I should have mentioned your name in the commit message.

Post by Jens Bauer

Post by Christophe Lyon

Post by Jens Bauer
* By default use twice as many cores for -jN as usual. (If you have 2 cores, use -j4)
-CPUS="`grep -c proces /proc/cpuinfo`"
+
+let CPUS=2*`getconf _NPROCESSORS_ONLN`

Using getconf is indeed more portable, I think.
However, I'm not sure we want the '2*' part of the patch,
since we sort-of rely on the -j factor in our validations
(e.g. number of independent builds in parallel on a
given machine)

Fair enough. :)
let CPUS=`getconf _NPROCESSORS_ONLN`
let cpucount=2*$CPUS
pmake="make -j$cpucount"; [ -w "$prefix" ] && smake="$pmake" || smake="sudo $pmake"
case `uname -s` in 'Darwin') isMac=1; smake="sudo $pmake" ;; 'Linux') isLinux=1 ;; esac
-I mentioned this in the support request as well. I'm creating two variables, which contains the code used for executing make. It has a 'built-in' -jN, where N is twice the number of cores.
-But if you're somehow querying make for the -j argument, then you'll probably need to use $CPUS directly.
(Normally -j2 will use around 50% CPU-time on a dual core machine - this is most likely due to the disk access involved, where the CPU waits for the disk to deliver data and sleeps until the data arrives).

This makes sense if you run only one build at a time on a machine, but
this is not our main use case.
For the time being, we try to optimize our validation bandwidth, and
we adjust the -j factor along with
the number of builds in parallel on a given build server.

As you may have guessed, when we run validations, we check several
targets, which happened
to be scheduled in parallel on several builders. We favor the
throughput in terms of validation
results. For instance, we prefer to have the results for 4 targets in
~2h rather than 3 targets
in ~1h30, then having to wait for another 1h30 for the next batch.
(Figures not accurate,
just to give you an idea). YMMV.

Thanks,

Christophe

Post by Jens Bauer
Love
Jens

[Jens Bauer]

Hi Christophe.

Post by Christophe Lyon

Post by Jens Bauer
You deserve some contributions/freebies, with all the hard work
you're already doing - thank you on behalf of thousands of people
using your toolchain!

- support for Darwin
- use of getconf

Awesome. I'll try it out. If I find any problems, I'll post to the list; otherwise you can assume things are alright. :)

Post by Christophe Lyon
Sorry, I realize that I should have mentioned your name in the commit message.

That's not necessary. I'm happy that I could contribute a little. What's important for me, is that I help improving stuff. =)

Post by Christophe Lyon

Post by Jens Bauer
-I mentioned this in the support request as well. I'm creating two
variables, which contains the code used for executing make. It has a
'built-in' -jN, where N is twice the number of cores.

This makes sense if you run only one build at a time on a machine, but
this is not our main use case.
For the time being, we try to optimize our validation bandwidth, and
we adjust the -j factor along with
the number of builds in parallel on a given build server.

Then you will need some 'breathing room' on those machines; I follow you.

Post by Christophe Lyon
As you may have guessed, when we run validations, we check several
targets, which happened
to be scheduled in parallel on several builders. We favor the
throughput in terms of validation
results. For instance, we prefer to have the results for 4 targets in
~2h rather than 3 targets
in ~1h30, then having to wait for another 1h30 for the next batch.
(Figures not accurate,
just to give you an idea). YMMV.

It makes perfect sense. =)


Love
Jens