Jump to content
TrinityCore
Amit86

[HowTo] Gdb-Backtrace Restarter With Auto Report E-Mail Sending

Recommended Posts

Hey all,

I wanted to write down a guide about preparing your linux server to be able to work as a gdb log tracer + restarter

I have been using it for around 2 years with no problem what-so-ever (other than the performance if having a compiled server with full debug flags on)

What is GDB?

GDB, the GNU Project debugger, allows you to see what is going on `inside' another program while it executes -- or what another program was doing at the moment it crashed.

What is Restater?

Restarter allows your core to crash, fall, get stuck and come back every time without anyone touching or doing anything manualy.

For this guide you acknowledge that you have basic knowledge in operating linux systems and using ssh as a cli

1.

First, lets make sure your Freeze Detector is set up corretly by editing the worldserver.conf file in `etc`

Find MaxCoreStuckTime and set it to 120

( MaxCoreStuckTime = 120 )

This will tell the server that if this demon is stuck for 2 minutes the core will crash automaticly leaving the restarter at work..

2.

Make sure you have sendmail installed

sendmail is a cli program that allows sending emails without any pre-setup etc

Try:

/usr/sbin/sendmail -f [email protected] [email protected]

or sendmail -f [email protected] [email protected]

If you get an email from [email protected] this means it works, if you dont -> Follow the sendmail installation guide which i found very helpful

3.

Go to your `bin` folder (where the worldserver bins are installed at)

create a new file - for the perpuse of the test call it `commands`

(nano commands)

put the following code in the file:

 

run -c ../etc/worldserver.conf

bt

bt full

info thread

thread apply all backtrace full

quit

Make sure that if you are using another path or other server config name, you have to alter it in the file. The rest, leave as is, its for the backtrace report to work. 4. Create crashes folder under bin mkdir Crashes the C is capsulated 5. creatre a new file, this file will be used as our restarter, mine is called gdbres.sh (nano gdbres.sh) In the file, put the following code:

 

while :

do

gdb -x=commands -batch ./worldserver > temp

tail -c 60000 temp | sed 1i"Subject: Crash Log" | /usr/sbin/sendmail -f [email protected] [email protected]

tail -c 60000 temp | sed 1i"Subject: Crash Log" | /usr/sbin/sendmail -f [email protected] [email protected]

mv temp Crashes/log$(date +\%Y-\%m-\%d-\%H-\%M-\%S).log

done

tail takes the last part of your `temp` file and sends it via mail, the -c 60000 chooses to take 60kb worth of text (which is the aprox gdb crash log filesize) and just send it trough

sed li = the subject of that email

the last command mv temp /Crashes/blabla moves the temp file to your crash log folder, if you dont have a crash folder it wont move the file and you wont have any log keepting so make sure you created the folder..

6.

Alter the files to be able to execute:

chmod +x commands

chmod +x gdbres.sh

7.

Starting the restarter is an easy job, personaly i use screen

if you are using screen do the following:

screen -A -dmS world ./gdbres.sh

This will start it in a new demon screen and you can forget about starting the core on your own for a while

if you are using simple cli commands you can use ./gdbres.sh &&

to make it start and forget about it

Notes:

  • Turning the restarter off gave me lots of problems when i wanted to close it, I started using webmin to be able to close it via their "Running programs"
  • Positive: To update core to tip etc, you can make install then just crash it. you dont have to fully shutdown the core only ctrl+c in the screen folder or shut it down via ingame and it will come back online using the restarter
  • Sometimes the gdb log comes back bigger than 60k, you can always go back to the Crashes folder and take the full crash.. Its usualy not bigger trough
  • To be able to debug your core with full debug flags you will have to use -DCMAKE_BUILD_TYPE=Debug to include additional debug-code in core in the cmake proccess as stated in the linux compile guide
  • Upvote 5

Share this post


Link to post
Share on other sites

Thanks for the information, to close restarter use (in console):

kill (pid)
show pid (is show before screen name):
screen -list[/code]


 

or use:

[code]screen -r world ctrl+c[/code]

A greeting and a lot of thanks

Edited by Dekadence

Share this post


Link to post
Share on other sites

Information about the registers is also useful in the crashlogs, you should add that

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×