Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

-static-libgcc is not working #98

Open
afxgroup opened this issue Jan 29, 2021 · 54 comments
Open

-static-libgcc is not working #98

afxgroup opened this issue Jan 29, 2021 · 54 comments

Comments

@afxgroup
Copy link

-static-libgcc used with -shared doesn't produce a file with libgcc linked statically and instead produce a file linked always with libgcc.so
Not only. When used with clib2 and shared objects dl* functions cause a crash because the .so files try to open newlib in some way and of course everything crash. If remove libgcc.so from gcc lib cure the problem

@raziel-
Copy link

raziel- commented Jan 29, 2021

The second part (When used with clib2 and shared objects dl* functions cause a crash because the .so files try to open newlib in some way and of course everything crash) sounds like it might be the same crash I'm getting with newlib (but I might ge completely wrong, of course)

@afxgroup
Copy link
Author

I have the same problem with newlib too.. on address 4. Which compiler are you using?

@raziel-
Copy link

raziel- commented Jan 30, 2021

@afxgroup

GCC 5.4.0 upto GCC 10.1.0 all produce the same crash with shared binaries for me.
So I assume it's something to to with the final binary accessing/handling libgcc.so and/or libstdc++.so.

But of course I dont really know, it might be something completely different.

@afxgroup
Copy link
Author

i had same problem in the past and usually i remove libgcc.so and use the static one. This is the only solution. However dl* function with the actual clib2 doesn't work. Maybe they had worked with old Exec and gcc version. but no more. I've found a workaround in my actual clib2 version but i have some problems and i don't like the fix. I'm trying to patch gcc and clib2 to work like newlib. But is hard..

@sba1
Copy link
Owner

sba1 commented Jan 30, 2021 via email

@afxgroup
Copy link
Author

mmm.. no. clib2.so will not work. It doesn't exists. But creating .so files and load them with dlopen/dlsym should. The problem is that clib2 is linking crtbegin when creating .so files with -shared flags and this cause problems. I've created shcrtbegin that contains __shlib_call_constructors and __shlib_call_destructors are used by InitSHLibs (at least autodocs says so). But i'm trying to patch amigaos.h file under rs6000 but is hard. Because while with newlib there is only a library it use "shared" and "!shared" statements. With clib2 there is multi lib and also strange cases with soft-float..

@sba1
Copy link
Owner

sba1 commented Jan 30, 2021 via email

@raziel-
Copy link

raziel- commented Jan 30, 2021

I would discourage the usgae of .so
altogether.

Big projects will soon start to be only available through cross-compiling then I'm afraid :-(

@afxgroup
Copy link
Author

Not only. For example the current (ancient) version of GemRB use plugins as .so files and it works without any problem. The problem is not in the .so files but how they are loaded. I've found the problem in clib2 for example and i'm trying to fix them.
However this issue is not related to .so files. This issue is related to a flag that is not working.

@raziel-
Copy link

raziel- commented Jan 31, 2021

Nice.
Is this bug also current in newlib and could it ge fixed there aswell?

I'll stand by for testing

@afxgroup
Copy link
Author

i don't know if it is related to newlib (and however i can't fix it since i'm not an os4 dev). However i have a small test case that show the problem. When i've something to test i'll tell you

@sba1
Copy link
Owner

sba1 commented Jan 31, 2021 via email

@afxgroup
Copy link
Author

Wait. I don't want a shared clib2. Clib2 can be static. And for clib2 i mean its libraries. lc lunix and lm.
But this doesn't means that i cannot generate a .so file and load it via dlopen.
In my branch i have a working example. The only problem is that the stdout/stderr/stdin is not working on loaded object. For the rest it is working correctly. I've tested it writing a file and works without any problem.
dlopen/dlsym are just wrappers to DLOpen and DLSym of exec library. And reading the autodoc it should map the loaded object in the main program memory. So why it should not work?
The problem instead is the crtbegin that is wrong in clib2 when creating shared objects. It should work like newlib shcrtbegin with two default functions are called by SHInit Exec function that are searching those functions with a fallback to _init and _fini. And here starts the problem on clib2.
However this is not a gcc problem. This is a clib2 problem.
This issue is just related to that switch in any case should not link against libgcc.so but against libgcc.a

@sba1
Copy link
Owner

sba1 commented Jan 31, 2021 via email

@afxgroup
Copy link
Author

well, yes. I'm speaking loading objects compiled with clib2. I know that if you try to open an .so compiled with newlib most probably will crash. But it is useful when opening plugins created with same clib. And sometime it happens if you port software from linux

@kas1e
Copy link

kas1e commented Feb 2, 2021

@sba1
@sodero
@agfxgroup
There is some mess started: We have now 4 different versions of clib2. One in Olaf's repo, another one in adtools, another one is a fork from Sodero with fixes about those "std:: not being a member blablalb". And the third one from agfxgroup. It is obvious that we start creating a mess now, and that is another reason why some users may think that newlib is better: it has just one single owner and everything in. No needs to think what/where, all just in the SDK will be.

@sba1
For now, I see Sodero make a pull request to adtools repo, with gcc9 "std::" fixes, but it used his clib2 fork. Will it be maybe better for example firstly accept his clib2 changes to adtools, and then accept pull requests to adtools itself? Then, changes from agfxgroup can be pull requested one by one too?

Because if we will continue to split stuff like this, then clib2 issue will be not only .so and things we discuss, but a fork-mess. Not the latest reason for this is the low speed of accepting pull-requests IMHO. For example, there is for almost 3 (!!) years pull request about a change to the proper, latest SDK link: #55 . Are there any reasons why you do not accept this pull request for 3 years?:)

@sba1
Copy link
Owner

sba1 commented Feb 2, 2021 via email

@obarthel
Copy link

obarthel commented Feb 2, 2021 via email

@sodero
Copy link
Contributor

sodero commented Feb 2, 2021

@sba1 Yes, I started small with a minor PR to not overwhelm @obarthel . We had an e-mail conversation where I got some feedback, leading to me updating the PR, but since then I haven't heard anything and I don't want to be too annoying. I also would like everything to end up upstream in Barthels repo, but whether that's realistic depends on what the scope of clib2 is. I understand that 68k and keeping it lean is key, but how lean, and if NG specific things are OK, is not clear to me.

@sodero
Copy link
Contributor

sodero commented Feb 2, 2021

@obarthel Yep, the problem we're discussing here is a very good one to have :-). If you feel that C++ > 98 is within the scope of clib2 then I can start splitting things up to avoid monster reviews. The only major things really missing from C++11/17/2a (using my fork) are the wchar stubs.

@afxgroup
Copy link
Author

afxgroup commented Feb 2, 2021

Clib2 use ancient code that on os4 is not a great solution. For example Seek that limits the files size to 32bit.
Adding #ifdefs everywhere is not a great solution for me.
I know that clib2 for 68k is used by Roadshow but i don't think that other software is using it. Forget my clib for now. I'm trying to clean it because i want something easy to implement, maintain and fix.
I've fixed the shared objects stuff for example that now is working correctly but this needs a change on gcc too because it use 2 different crtbegin and crtend linked with -fPIC and with different code.

The clib2 code is from 2005. So 4.0 era. It needs to be changed but if you want to maintain the os3 compatibility is hard and the code will be a mess. (imho of course). That's why i've started to erase all 68k code. Simply because no one is using it (except roadshow that however can be compiled with the original clib2).

So, in the end. Forget my clib2 version. Use the original one in gcc but keep in mind that something doesn't work if you don't change it deeply

@kas1e
Copy link

kas1e commented Feb 2, 2021

Use the original one in gcc but keep in mind that something doesn't work if you don't change it deeply

And as seems, clib2 wasn't targeting at all to be "modern clib for os4" for everyone, but mostly for personal Olaf's needs, then it again proves our fear that only newlib is left (or, different forks of all kinds). I asked Hyperion's management about opening newlib, and has been told that will be considered later. But it also means chances are low, IMHO.

@obarthel
Are you want to keep clib2 be the same "old" without improvements, or want to improve it further? If later, things need to be changed deeply for sure. And 68k code in only stop it from progress.

@sba1
Copy link
Owner

sba1 commented Feb 2, 2021 via email

@sodero
Copy link
Contributor

sodero commented Feb 2, 2021

I don't really see 68k as a big problem. There's an upside to it as well, keeping 68k means more users and users are the best testers. Of course this depends on the code base being kept together as much as possible. There's just three options here, relying on newlib (where we can't fix anything at all), forking clib2, making it a pure OS4 lib, which makes things easier short-term but might lead to cherry-picking and confusion in the long run, and improving clib2 as it is now (if @obarthel thinks that's ok). I don't just think that the last option is the least bad one, I actually think it's a good one.

@obarthel
Copy link

obarthel commented Feb 2, 2021 via email

@obarthel
Copy link

obarthel commented Feb 2, 2021 via email

@obarthel
Copy link

obarthel commented Feb 2, 2021 via email

@sodero
Copy link
Contributor

sodero commented Feb 2, 2021

@obarthel I also feel a bit uneasy when I see wchar* (especially if Amiga shows up in the same sentence). I have no clue how to test the clib plumbing in a comfortable way. As it is now, the stubs don't affect libstdc++ since there's not enough of them :-) (which means we don't trick autotools into enabling features that aren't there).

@sodero
Copy link
Contributor

sodero commented Feb 2, 2021

Since the gcc's 68k backend has been modernized in the meantime it may be worthy to look at it again. Whether modern C++ applications can run on a 68k is different question though ;)

@sba1
Indeed, in gcc10 things that affect our 68k backend have happened. OS4 is not a problem, but I think I need your help with one of the 68k patches :-) (once we've resolved the clib2 issue)

@afxgroup
Copy link
Author

afxgroup commented Feb 2, 2021

There are really too many changes to keep both 68k and os4 and maintain the code readable.
One change i've made is to use ExamineData instead of FileInfoBlock.
This because the old functions now on OS4 are deprecated. It was a pain. And, yes, you could wrap everything around #ifdefs but it is really unreadable also with modern editors and it is hard to fix also problems.

Another change was to replace old Seek calls with new OS4 functions that supports 64bit. Again you could use some #define to support both platforms but is hard because also return values are changed. It is no so easy. really.
That's why i told you to continue to support classic clib2 version but keep in mind that is really hard to fix things with both platforms in mind.

Also shared objects, for example, cannot be used on OS3 and needs two different begin/end files and not crtbegin/crtend.

Remember that on OS4 you have ixemul that is perfect to port software from *nix. That's why i still prefer to remove 68k code. But nothing prevent to get my code where possible and put it into Olaf's clib2

@sodero
Copy link
Contributor

sodero commented Feb 2, 2021

@afxgroup You can always #ifdef an entire function (within reason of course) if it becomes to complex. It might also be worth it to look at how these type of 'transitional' problems are solved on other platforms. In this case, 32/64 bit problems, fseek, fseek64 and so on. But take this with a shovel of salt, I haven't seen the code so it's very easy for me to pretend that it's doable ;)

@sodero
Copy link
Contributor

sodero commented Feb 2, 2021

One way of managing variability could be to use a well thought out directory structure. It's very flat as it is now, there are pros and cons to this of course.

@kas1e
Copy link

kas1e commented Feb 2, 2021

@everyone
And can anyone explain why we need to keep those "thread-safe" and "usual" versions of libs? Why it all just not "thread-safe" by default of a single version and that it?

@sodero
Copy link
Contributor

sodero commented Feb 2, 2021

@kas1e
No point in being thread safe when you're not doing any threading :) The thread safe builds use semaphores in low level stdlib functions like malloc and so on. I have no idea how expensive that is. But it's in places where cost matters.

@afxgroup
Copy link
Author

afxgroup commented Feb 2, 2021

One way of managing variability could be to use a well thought out directory structure. It's very flat as it is now, there are pros and cons to this of course.

This means separate code.. and have the same result. It is always possible use code from one clib2 to the other one (and viceversa). But i don't like mess code. Is just my way to do things.
For example. there is a fix on timer funcions can be easily ported to os3 version.
I've "copied" your complex functions and adapted to my version. Why we can't work in this way?

@sodero
Copy link
Contributor

sodero commented Feb 2, 2021

@afxgroup Sure, it's open source, forking is an option. But the spark in this discussion was kas1e:s fear that forks will create confusion and I agree with him. It's just a matter of opinion though. And about the directory structure; I didn't mean that we should just throw everything in a bucket, that would be pointless, just like you said.

@obarthel
Copy link

obarthel commented Feb 3, 2021 via email

@obarthel
Copy link

obarthel commented Feb 3, 2021 via email

@obarthel
Copy link

obarthel commented Feb 3, 2021 via email

@sodero
Copy link
Contributor

sodero commented Feb 3, 2021

Am 02.02.21 um 19:30 schrieb Ola Söder:
Since the gcc's 68k backend has been modernized in the meantime it may be worthy to look at it again. Whether modern C++ applications can run on a 68k is different question though ;) @sba1 https://github.com/sba1 Indeed, in gcc10 things that affect our 68k backend have happened. OS4 is not a problem, but I think I need your help with one of the 68k patches :-) (once we've resolved the clib2 issue)
This may sound quaint, but the 68k AmigaOS platform needs a native modern 'C' compiler and a runtime library which is designed to work with the actual operating system instead of pretending that it's actually a 1990'ies BSD Unix. Furthermore, that 'C' compiler should be a 68k native 'C' compiler, and not a cross-compiler. This is where clib2 can help. 68k AmigaOS development currently uses the old tool stack (SAS/C) and it can't do so forever.

-- Home: Olaf Barthel, Gneisenaustrasse 43, D-31275 Lehrte Net: [email protected] (Home), [email protected] (Work)

Luckily 68k wasn't killed of in g cc10, but it was a close call. I don't know who did the work but there's still hope for a fresh native compiler. Someone also started the work on a LLVM backend for 68k which is nice. That code base is a lot easier to understand so in the long run it might be interesting. If upstream are willing to accept a new legacy backend is another question, but even if they don't, it should be easier for us to maintain an out of tree 68k backend. I've been thinking of doing some experiments but 68k is not my cup of tea to be honest and I know nothing about the hunk format. It would be fun though, but probably a lot of work.

@sodero
Copy link
Contributor

sodero commented Feb 3, 2021

@obarthel I opened two more PRs by the way, bugfixes that should be useful to everyone.

@obarthel
Copy link

obarthel commented Feb 3, 2021 via email

@obarthel
Copy link

obarthel commented Feb 3, 2021 via email

@obarthel
Copy link

obarthel commented Feb 3, 2021 via email

@obarthel
Copy link

obarthel commented Feb 3, 2021 via email

@sba1
Copy link
Owner

sba1 commented Feb 3, 2021

How about moving this discussion to somewhere else? Github discussions at our adtools group comes into my mind. This would be visible then to @adtools/developers as well. I would be glad to also add others to that organization.

@kas1e
Copy link

kas1e commented Feb 3, 2021

@sba1
Discussions are good, but better to review and accept/reject pool requests, IMHO :) Those discussions always take time and energy instead of the actual stuff.

Btw, is there any reason why you didn't accept for few years fixed SDK URL to download the latest one from Hyperion? Maybe there were some bugs because of which you revert to older SDK while it has strangely 53.30 name, but download instead older one?

@sodero
Copy link
Contributor

sodero commented Feb 3, 2021

@sba1

How about moving this discussion to somewhere else? Github discussions at our adtools group comes into my mind. This would be visible then to @adtools/developers as well. I would be glad to also add others to that organization.

Fair point. I'd really to join if that's OK.

Just one last comment to @obarthel , then I'm out :-) (I feel like a child saying that).

The modern GCC still uses binutils for binary file wrangling, doesn't
it? The Amiga-aware binutils are still being maintained, albeit not
necessarily in the context of adtools. Gunter Nikl still maintains his
binutils, specifically the Amiga-hunk support.

Yes, binutils is still used, good to know that it's still maintained. I was refering to LLVM though. I know very little about what happens behind the frontend, most books focus on that (which is understandable). The Plauger book looks nice BTW, pdf downloaded (seems a bit hard to come by a real one).

@obarthel
Copy link

obarthel commented Feb 5, 2021 via email

@kas1e
Copy link

kas1e commented Feb 7, 2021

@obarthel
@sba1
Giving the VERY unacceptable and slow pool request reviews and accepting or rejecting them, you of course kill all sodero's motivation to continue with it :( Waiting weeks to accept simple patches is unacceptable and of course, no one can works fast when accepting/reviewing/rejecting pool requests taking months. He many times repeat what he waiting for, and even push-request his changes few times, but nothing.

So, as I result, how we can be in hope that clib2 will replace anything ?:) Of course, it may, but will not if we continue like this.

Probably the best we can do now, it only improves GCC for newlib then (As I told you from the beginning), because and clib2 is not in the scope of change anything, and Olaf has no time for it. And it just looks mostly like a library for Olaf's needs. So...

But then, even with "just" GCC fixes, it also means Sebastian need to find time on accepting/review request. If there is no time on Sebastian's side, then probably better to made a fork for our local needs, indeed. And maybe later we can ask Hyperion to include to official SDK our fork, and ask all developers to use this one instead.

Because no progress sucks hard and kills all motivation. It's now all the same as with newlib : we wait for someone to find time to accept/review changes, the same as we want from Hyperion to release updated newlib (on which at least Frederik works quite heavy all the time and updating it).

I mean, we lucky to have interest from Sodero now, who actually do something on that front, why we should kill all his motivation with such an unacceptable speed of just pure reviewing/accepting/rejecting patches?

@sodero
Just to not lost interest and motivation, maybe forget about adtools/clib2 repos for now, and keep working on a local fork with gcc9/gcc10? I can help with all tests, etc, and later we can ask Hyperion to include this one instead. Or that all will never end. Begging for months and cry like a child in hope that Sebastian and Olaf will find time to review and accept or reject the poll-request suck very much and unacceptable.

@obarthel
Copy link

obarthel commented Feb 7, 2021 via email

@kas1e
Copy link

kas1e commented Feb 7, 2021

@obarthel
Yeah, that understandable of course... Maybe indeed better to make some kind of fork then, and working on, and later when you will have more time maybe review/integrate it, or maybe not, but at least work will be not stopped in hope to be merged?

@sba1
Copy link
Owner

sba1 commented Feb 7, 2021 via email

@sodero
Copy link
Contributor

sodero commented Feb 7, 2021

@kas1e, @obarthel and @sba1

I think the whole point of having a hobby is to do something that is as far away from work as possible and doesn't involve stress. When I wrote on amigans.net that I'll halt work on gcc for now (and I really meant for now, not forever), or atleast slow down, that was what I had in mind, I really didn't want to stir up any drama. Having a heap of PR:s waiting for review is stressful, and having a fork repo so full of branches that it starts to look like the London subway map is unpleasant, so I think it's in the interest of everyone, me included, to slow down. Please don't interpret this as a symptom of frustration, it's about finding a balance that's ok for everyone. A short-term pragmatic approach would be to have a branch in adtools where we could use a clib2 fork. In that way I could keep on trucking with gcc without generating a ton of pr:s while at the same time having the gcc patches in one place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants