Skip to content

Conversation

@Naios
Copy link
Contributor

@Naios Naios commented Mar 9, 2015

Use Type safe variadic templates for for toplevel api functions (TC_LOG & PSendSysMsg)

  • make use of perfect forwarding to increase speed.
  • improve performance of Appender::write by using std::ostringstream && std::move

It will show error messages if you use wrong formattters in logging functions ("%s" instead of "%u" for example).
Its also possible to use std::string as formatter now what makes it unnecessary to call c_str() on it.

Also there is a global Trinity::StringFormat function added that allows you to format strings in any way.

There is also the possibility to convert more vaargs to type safe variadic templates if desired (mysql query api, internal format calls)

ToDo:

  • Memory leak profiling -> No leaks detected by visual leak detector.
  • Static analysis
  • Find wrong formatters

-> Finished

@xurxogr
Copy link
Contributor

xurxogr commented Mar 9, 2015

Any reason to move implementation of some functions from .cpp to .h? if there is no improvement (by profiler) they should remain at .cpp files (+ adding implementation to .h ends up adding unnecessary includes to the rest of the code due to include chain)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are arguments evaluated even if sLog->ShouldLog() returns false ? if so, then this is not an acceptable change due to performance reasons

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reverted this change, my mistake.

@jackpoz
Copy link
Member

jackpoz commented Mar 9, 2015

could you elaborate more on the need of using shared_ptr ?

@Naios
Copy link
Contributor Author

Naios commented Mar 10, 2015

@Spp- because of variadic templates i'm sure it will lead to linker errors if i put it into the .cpp file.
@jackpoz I'm using unique_ptr with std::move now. In my opinion it is better to show who will take over the pointer instead of just create the object on heap and pass it through. Sure it's not needed here but it helps to keep the code clean.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will lead to random characters in front of the string.
Anyone got an idea?

@Naios
Copy link
Contributor Author

Naios commented Mar 10, 2015

Strange linker errors on clang:

�[0m../game/libgame.a(WorldSocketMgr.cpp.o): In function WorldSocketMgr::StartNetwork(boost::asio::io_service&, std::string const&, unsigned short)': /home/travis/build/TrinityCore/TrinityCore/src/server/game/Server/WorldSocketMgr.cpp:(.text+0x26f): undefined reference toboost::asio::socket_base::max_connections'

@Naios Naios force-pushed the typesafelog branch 2 times, most recently from d2c7d2f to 95a9b22 Compare March 10, 2015 10:05
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this should be put in Trinity namespace (also function naming standards...)

@Naios Naios changed the title [WIP] Type safe logging Type safe logging Mar 10, 2015
@jackpoz
Copy link
Member

jackpoz commented Mar 10, 2015

With "It will warn you if you use the wrong formatter" what exactly do you mean ?
I see the description has been edited with "Its possible that there are exceptions thrown because of invalid formatters", does it mean this PR adds unhandled exceptions that cause a crash ? Isn't the point of this PR to ensure the correct formatting at build time ? Otherwise is just the same as dynamic analysis.

If on the other hand the exception is caught, where and how is the information about the line triggering the exception saved to fix the issue ?

@Naios
Copy link
Contributor Author

Naios commented Mar 10, 2015

You are right, as i started to work on this i thought its possible to catch it at build time but its only possible to catch it at runtime because the format is read at runtime only.
If the formatter or format is wrong it will throw an exception on runtime that will go through until:

try
    {
        return mainImpl(argc, argv);
    }
    catch (std::exception& ex)
    {

The actual problem is that (for me) the WeathyException stracktrace isn't shown on crash.
If i want to find out what formatter is producing the exception i need to debug in msvc.

@jackpoz
Copy link
Member

jackpoz commented Mar 10, 2015

WEH not showing the stacktrace sounds like a bug, could you open an issue so I can look into that ? feel free to reference this PR to reproduce it.

As for the log format, this is my idea of format errors handling, listed by "what I'd like to happen" on top:

  • build time error <-- currently provided by static analysis/coverity
  • runtime logged error with source line that caused the issue, worldserver keeps running without any issue <-- currently provided by dynamic analysis/valgrind
  • crash <-- current behavior if using %s with a integer as input

The current implementation makes things worse for users who don't really care much about logs, with "always crashing" instead of "crashing sometimes". It might be worth to investigate about different ways to handle the errors/different ways to catch them at build time.

@vitaut
Copy link

vitaut commented Mar 10, 2015

If you only use printf-like format strings and OK with macros, then you can catch format type errors at compile time using the technique described in fmtlib/fmt#62 (comment).

@Naios
Copy link
Contributor Author

Naios commented Mar 10, 2015

@vitaut Thanks for your tip, i will try it out.
@jackpoz opened an issue here #14329.
I will try to catch the exception and log an error with the line number and source file or something to correct the format.

@Naios
Copy link
Contributor Author

Naios commented Mar 10, 2015

@jackpoz see:
screenshot_2

@Naios Naios force-pushed the typesafelog branch 3 times, most recently from 0dc97dc to eed512f Compare March 11, 2015 09:06
* improves safety and log speed through:
  - variadic templates
  - perfect forwarding
* fixes a newline in db logs
* improve performance of Appender::write by using std::ostringstream && std::move
@DDuarte
Copy link
Contributor

DDuarte commented Mar 18, 2015

👍 :shipit:

Carbenium added a commit that referenced this pull request Mar 18, 2015
@Carbenium Carbenium merged commit cc0c9ad into TrinityCore:6.x Mar 18, 2015
@Naios
Copy link
Contributor Author

Naios commented Mar 18, 2015

If requested i can add type safety also to SQL query api functions like PQuey and its derivates and other functions that use still vaargs and char[] with a very high field size (to increase speed).

@DDuarte
Copy link
Contributor

DDuarte commented Mar 18, 2015

That would be quite nice indeed

Carbenium added a commit to Carbenium/TrinityCore that referenced this pull request Mar 18, 2015
Type safe logging
(cherry picked from commit cc0c9ad)

Conflicts:
	src/server/bnetserver/CMakeLists.txt
	src/server/game/Server/WorldSocketMgr.cpp
	src/server/shared/Logging/AppenderDB.cpp
	src/server/worldserver/CMakeLists.txt
@ariel-
Copy link
Contributor

ariel- commented Mar 18, 2015

After this applied, .gps command crashes server

@DDuarte
Copy link
Contributor

DDuarte commented Mar 18, 2015

Post the crash/error log

@ariel-
Copy link
Contributor

ariel- commented Mar 19, 2015

Researched it a little in Debug mode:

Error raised trying to retrieve argument nº 17 (LANG_MAP_POSITION string requires 23 arguments) and the default is set to an arbitrary 16 in the class ArgList used in format.h, line 1053, just bump the limit a little.

After doing this, I get another exception, I setted MAX_ARGS to 32.

Log: https://gist.github.com/ariel-/d9582c6450cdbc8c1c92#file-e96676aeddb4_worldserver-exe_-18-3_22-18-38-txt

@Shauren
Copy link
Member

Shauren commented Mar 19, 2015

You cannot simply change a random value in a library and expect it to work...
But you can make a pull request that splits trinity_string id 101 into 2 strings with smaller amounts of arguments

@ariel-
Copy link
Contributor

ariel- commented Mar 19, 2015

True that, maybe the maximum is enforced elsewhere, the problem is, that this library should provide same funcionality than vsprintf, rather than possibly forcing a change of those trinity_strings with more than 16 format placeholders.

@Shauren
Copy link
Member

Shauren commented Mar 19, 2015

Already checked for this, gps string is the only case

@Naios
Copy link
Contributor Author

Naios commented Mar 19, 2015

The question is: do a Log or PSensSysMsg call really need more then above 16 args? I think its not needed.
LANG_MAP_POSITION is the only string (hardcoded and in db) that takes more then 16 args so we could just split it up.

@Shauren checked it yesterday too in core and db its only LANG_MAP_POSITION .

EDIT: I'm on it.

@Naios Naios deleted the typesafelog branch March 19, 2015 08:39
@ariel-
Copy link
Contributor

ariel- commented Mar 19, 2015

#14405

M0ra added a commit to M0ra/Core that referenced this pull request Mar 22, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants