Q: reasons for lack of support for C++ in OP-TEE?

Discussion:

Godmar Back

2017-08-11 13:53:26 UTC

In a recent github thread
<https://github.com/OP-TEE/optee_os/issues/1708#issuecomment-320245973> it
was suggested that I ask this list about what the exact reasons for the
lack of C++ support are, and how/if they break down by C++ feature so as to
gauge a possible investment in remedying this situation at least partially.

In other words, suppose I changed the build process to include libstdc++,
libgcc, and libgcc_eh (and/or other runtime support commonly linked with
C++ programs), what features of C++ would work/still fail? And what
implementation work would be required to implement the missing features?
On a related note, are there intrinsic properties of the secure environment
that may conflict with running C++ code, if any?

Thank you.

- Godmar

Jim Wilson

2017-08-11 20:04:36 UTC

Permalink

Answering this question would required detailed knowledge of OP-TEE,
and I doubt that anyone on this list has such knowledge. I was just
reading some FAQs about OP-TEE myself.

I would guess that part of the problem is that libstdc++ is so large,
and isn't designed to be pulled apart into small pieces, as there are
a lot of features that interact with other features. Just running nm
on libstdc++.so, and grepping for GLIBC_, I see that it calls 105
glibc functions. Some of these functions are I/O related and are
obvious problems, like fopen, fclose, fread, fwrite. Some of them are
clock related and could be an issue, like clock_gettime, gettimeofday.
The string and memory routines are probably OK, but libstdc++ may be
using more of them than OP-TEE implements. There are also a lot of
wide-character and multi-byte character routines being used that may
not be available. There are pthread related functions that may or may
not be a problem. There are also some environment and locale related
routines that are likely a problem, but maybe you can just provide a
default environment and locale.

Otherwise, I don't know enough about OP-TEE to guess what might be a
problem. The exception handling support just requires the ability to
read your own process memory, including the stack, the unwind info,
and the ELF headers that get loaded into memory at program start. I
don't know if that is a problem for OP-TEE.

Jim

Godmar Back

2017-08-11 20:34:58 UTC

Permalink

Post by Godmar Back

Post by Godmar Back
In a recent github thread
<https://github.com/OP-TEE/optee_os/issues/1708#issuecomment-320245973>

Post by Godmar Back
was suggested that I ask this list about what the exact reasons for the
lack of C++ support are, and how/if they break down by C++ feature so as

Post by Godmar Back
gauge a possible investment in remedying this situation at least

partially.
Answering this question would required detailed knowledge of OP-TEE,
and I doubt that anyone on this list has such knowledge.

Is this the wrong mailing list to ask the question then?

Post by Godmar Back
I would guess that part of the problem is that libstdc++ is so large,
and isn't designed to be pulled apart into small pieces, as there are
a lot of features that interact with other features. Just running nm
on libstdc++.so, and grepping for GLIBC_, I see that it calls 105
glibc functions. Some of these functions are I/O related and are
obvious problems, like fopen, fclose, fread, fwrite.

Normally, libraries are decomposed into individual .o files to avoid
exactly this phenomenon where an undefined reference to function X pulls in
definitions of functions Y, Z, etc. which then cause a chain of
dependencies to other .o files the linker must resolve. Is libstdc++
different in this respect?
Because otherwise, features would only be pulled in if required, and that's
a much more manageable problem.

- Godmar

Jim Wilson

2017-08-11 21:12:31 UTC

Permalink

Post by Godmar Back
Is this the wrong mailing list to ask the question then?

Perhaps, but maybe you will get a response on Monday from someone in
Europe that didn't see it today.

Post by Godmar Back
Normally, libraries are decomposed into individual .o files to avoid exactly
this phenomenon where an undefined reference to function X pulls in
definitions of functions Y, Z, etc. which then cause a chain of dependencies
to other .o files the linker must resolve. Is libstdc++ different in this
respect?
Because otherwise, features would only be pulled in if required, and that's
a much more manageable problem.

Yes, of course, if you static link, then you only get the .o files
that you need. But as I mentioned before, a lot of C++ features
depend on other C++ features. A simple hello world program compiled
and statically linked is over 2MB.

palantir:2089$ cat tmp.cc
#include <iostream>

int
main (void)
{
std::cout << "Hello world\n";
return 0;
}
palantir:2090$ g++ -static tmp.cc
palantir:2091$ ls -lt a.out
-rwxrwxr-x 1 wilson wilson 2181416 Aug 11 14:03 a.out
palantir:2092$

We can see all of the object files being pulled in if we pass the
--verbose option to the linker. There are 92 of them.

palantir:2092$ g++ -static tmp.cc -Wl,--verbose | grep libstdc
attempt to open /usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a succeeded
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ios_init.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)globals_io.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ios-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ext11-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ostream-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)wlocale-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ios.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-wstring-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)istream-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)functexcept.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-stdexcept.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)streambuf-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_throw.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_exception.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_personality.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)del_op.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_catch.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_aux_runtime.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)bad_typeid.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)bad_cast.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)si_class_type_info.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_call.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)new_opv.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)pure.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)tinfo.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)class_type_info.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)new_op.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)new_handler.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)vmi_class_type_info.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)del_opv.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_alloc.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_terminate.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_globals.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)bad_alloc.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)dyncast.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)locale.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)istream.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)locale_facets.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)time_members.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)locale_init.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)stdexcept.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)c++locale.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)monetary_members_cow.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)numeric_members.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)basic_file.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ios_locale.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)streambuf.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)messages_members_cow.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)numeric_members_cow.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)codecvt.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-istream-string.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)codecvt_members.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)collate_members_cow.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ctype.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)string-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cxx11-locale-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)locale-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cxx11-wlocale-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-string-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)snprintf_lite.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)fstream-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cxx11-stdexcept.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)system_error.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-shim_facets.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cow-locale_init.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ctype_configure_char.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)functional.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)ctype_members.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cxx11-shim_facets.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)wstring-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)future.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)regex.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)lt1-codecvt.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cxx11-ios_failure.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)iostream-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)compatibility.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)bad_array_new.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)guard.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_unex_handler.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_ptr.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_term_handler.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)monetary_members.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)misc-inst.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)istream-string.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)collate_members.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)messages_members.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)futex.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)condition_variable.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)guard_error.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)vterminate.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)eh_type.o
(/usr/lib/gcc/x86_64-linux-gnu/5/libstdc++.a)cp-demangle.o
palantir:2093$

Perhaps there are other things that can be used that have smaller
requirements, but in general, the C++ library is not designed for
small slices. Everything depends on everything else, and it is best
to use shared linking and use the whole thing as a single unit.

Jim

Michael Zimmermann

2017-08-12 05:27:34 UTC

Permalink

That also depends on what you want to use C++ for.
If you just want to use the convenient syntax and language features
then you don't need to link against any additional library.
In that case you'd disable exceptions and rtti and implement
new/delete using malloc/free.

Thanks
Michael

Post by Godmar Back
In a recent github thread
<https://github.com/OP-TEE/optee_os/issues/1708#issuecomment-320245973> it
was suggested that I ask this list about what the exact reasons for the
lack of C++ support are, and how/if they break down by C++ feature so as to
gauge a possible investment in remedying this situation at least partially.
In other words, suppose I changed the build process to include libstdc++,
libgcc, and libgcc_eh (and/or other runtime support commonly linked with
C++ programs), what features of C++ would work/still fail? And what
implementation work would be required to implement the missing features?
On a related note, are there intrinsic properties of the secure environment
that may conflict with running C++ code, if any?
Thank you.
- Godmar
_______________________________________________
linaro-toolchain mailing list
https://lists.linaro.org/mailman/listinfo/linaro-toolchain

Jeffrey Walton

2017-08-12 05:55:21 UTC

Permalink

On Sat, Aug 12, 2017 at 1:27 AM, Michael Zimmermann

Post by Michael Zimmermann
That also depends on what you want to use C++ for.
If you just want to use the convenient syntax and language features
then you don't need to link against any additional library.
In that case you'd disable exceptions and rtti and implement
new/delete using malloc/free.

Instead of pounding a round peg into a square hole with C++, maybe the
path to pursue is GCC support for try/finally in C. You get most of
the semantics you are looking for, and someone as experienced as you
will know exactly what to do with the feature.

Lack of try/finally support means library often do awful things.
Asterisk was full of issues, like resource leaks, that try/finally
would have fixed. Eventually Asterisk moved to nested/inner functions
and trampolines. They silently lost NX stacks, which was a cure as bad
as the disease. It just moved problems around rather than fixing them.

Microsoft compilers support try/finally in C. I recall a GCC bug
report years ago asking for it, but it was declined.

Jeff

Godmar Back

2017-08-12 14:42:07 UTC

Permalink

I'm sorry the discussion got off track about the relative virtues of C++
for embedded programming.

Our need is to run parts of existing C++ code (the px4 <http://px4.io/>
library for drones) under OP-TEE. The C++ features we need support for thus
are the ones used there, and we'd like to change it as little as possible.

So far, I've heard the risk that C++ binaries get large if you include
libstdc++ features such as cout. Ok. See my footnote (*) below.

Size aside, what other properties of OP-TEE are risking conflict with C++'s
runtime that anyone already knows of?

- Godmar

(*): In this context, I've found a FAQ entry describing why they have such
large object files
<https://gcc.gnu.org/onlinedocs/libstdc++/faq.html#faq.size> and mentioning
a garbage collection feature of the GNU linker, which I hadn't heard of. It
reduces the size for the static Hello World program from a roughly 1.7M
text section to 1.3M text:

$ cat hello.cc
#include <iostream>

int
main()
{
std::cout << "Hello, World\n";
}

$ g++ -static hello.cc -o hello
$ size hello
text data bss dec hex filename
1711137 33580 18056 1762773 1ae5d5 hello
$ ls -l hello
-rwxrwxr-x 1 gback gback 2181416 Aug 12 10:34 hello

$ g++ -static -Wl,--gc-sections hello.cc -o hello
$ size hello
text data bss dec hex filename
1321767 24388 16424 1362579 14ca93 hello
$ ls -l hello
-rwxrwxr-x 1 gback gback 1567592 Aug 12 10:34 hello

Post by Jeffrey Walton
On Sat, Aug 12, 2017 at 1:27 AM, Michael Zimmermann

Instead of pounding a round peg into a square hole with C++, maybe the
path to pursue is GCC support for try/finally in C. You get most of
the semantics you are looking for, and someone as experienced as you
will know exactly what to do with the feature.
Lack of try/finally support means library often do awful things.
Asterisk was full of issues, like resource leaks, that try/finally
would have fixed. Eventually Asterisk moved to nested/inner functions
and trampolines. They silently lost NX stacks, which was a cure as bad
as the disease. It just moved problems around rather than fixing them.
Microsoft compilers support try/finally in C. I recall a GCC bug
report years ago asking for it, but it was declined.
Jeff

Peter Smith

2017-08-14 09:27:04 UTC

Permalink

Hello Godmar,

This is a very difficult question to answer as it will depend a lot on
the constraints of the OP-TEE environment and what subset of the C++
language and libraries that the developers are willing to support. I
can speak a little bit about generic support of C++ in embedded
systems via my experience with ARM's commercial toolchain, which is
based on a mixture of clang, libc++, libc++abi and libunwind; and
ARM's own C-library and linker. This isn't going to directly map to
the GNU world but I don't think it will be too far off.

The minimal additional support for C++ over C is support for the
language runtime, representing helper functions that the compiler
expects to be defined, regardless of whether these are called in the
source code; for example the various forms of new and delete. These
functions often have a cxa prefix are defined generically by the
Itanium C++ ABI and built upon by platform specific ABIs such the ARM
C++. In the llvm world these are defined by the libc++abi library with
stack unwinding provided by the libunwind library. In general I
wouldn't expect there to be any major difficulty in providing such a
run-time support library for OP-TEE, but I don't know anything at all
about any special restrictions that platform has that might get in the
way. For example I don't know whether OP-TEE requires additional
libraries to be certified?

Any use of C++ static constructors need to be initialised by the
run-time environment, this is typically handled by the C-library
start-up.

Support for the C++ standard library is more complicated, particular
with C++11 and beyond. There are some areas of functionality that are
fairly easy to say this won't work on our platform. For example
std::thread may not work if the underlying platform has no support for
threads. There are some areas such as the containers and algorithms
that will require little additional support beyond the runtime. Some
features such as use of iostream will significantly bloat the code as
this will pull in all the locale support, require static
initialisation of the cin, cout objects.

Perhaps the most difficult decision is whether to support exceptions.
For your own code, choosing not to support exceptions isn't a problem
as both gcc and clang can compile without exception tables, however
many C++ libraries, including the standard library make use of
exceptions, and an uncaught exception results in program termination.
The libc++ library can be compiled without exceptions but this just
means it will terminate the program if an exception would be thrown.

To summarise, if the platform runtime support is limited, there isn't
an easy answer to say C++ is supported, and the answer heavily depends
on the design choices of the platform holder. The OP-TEE developers
would most likely have to define a subset that is supported and
constantly monitor it as toolchains change. They may not consider this
worth the trade-off against the simpler answer of C++ is not
supported.

Hope this is of some use

Peter

Renato Golin

2017-08-14 09:53:49 UTC

Permalink

This post might be inappropriate. Click to display it.