Project

General

Profile

Support #456

Crash during generation of webpage for documentation

Added by Nikhil Jain over 5 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
High
Category:
-
Target version:
-
Start date:
03/31/2014
Due date:
% Done:

0%

Spent time:

Description

Following type of errors were seen:

s.unwrap()
TypeError: 'NoneType' object is not callable

The code seems correct, but still complains.

History

#1 Updated by Nikhil Jain about 5 years ago

  • Priority changed from Normal to High

#2 Updated by Nikhil Jain about 5 years ago

I spent a few hours on this, but with no success. beautifulsoup seems to be returning NoneType (or corrupt data). Python lovers..anyone?

#3 Updated by Nikhil Jain about 5 years ago

  • Status changed from New to In Progress

#4 Updated by Zhengqi Yang about 5 years ago

Where can I find the code of this?

#5 Updated by Nikhil Jain about 5 years ago

Its distributed with Charm:

charm/doc/charm++ : make html

#6 Updated by Zhengqi Yang about 5 years ago

Compiles just fine on my laptop, but I got the same error on a lab machine. What versions of python and beautifulsoup are you using?

Worked with python 2.7.6 and beautifulsoup 4.2.1.

#7 Updated by Zhengqi Yang about 5 years ago

wrap() method is not implemented until Beautiful Soup 4.0.5, so upgrading beautifulsoup4 package should fix this. (http://www.crummy.com/software/BeautifulSoup/bs4/doc/#wrap).

Just checked that the current version of bs4 on lab machines is 4.0.2.

#8 Updated by Nikhil Jain about 5 years ago

  • Status changed from In Progress to Resolved
  • Assignee set to Zhengqi Yang

Great work. Thanks.

#9 Updated by Michael Robson about 5 years ago

Has this fix actually been implemented?

#10 Updated by Nikhil Jain about 5 years ago

No, I marked it resolved since the documentation is not buggy. We still need to update it on one of the machine and set up the cron job. Can you take a shot at it on panache?

#11 Updated by Michael Robson about 5 years ago

Sure. Looks like there is a nightly Cron job to ensure that it builds correctly (in /etc/cron.daily/charm_manupdate) but it appears to delete the results. Whatever happened to the Gerrit hook we set up?

#12 Updated by Nikhil Jain about 5 years ago

Which Gerrit hook?

#13 Updated by Nikhil Jain about 5 years ago

It might be doing a clean up because of failure. Remember, we first need to update bs to a higher version.

#14 Updated by Michael Robson about 5 years ago

A while ago we tried to import the old post-commit (or maybe push) hook from Gitosis into Gerrit which automatically built the manual. I don't remember if that actually got ported or we decided it wasn't worth it. Regardless, I'd like to look at that hook to know what it was doing to build the manual.

The current cron script I found just tests to see if the manual builds without failing, it doesn't actually update the manual. In fact, this script may be broken (or just disabled) since I haven't seen any emails to the sysadmin list which it should be emailing every time the build fails.

#15 Updated by Nikhil Jain about 5 years ago

I dont think we got the Gerrit based setup working. I would ignore it for now and just make sure that cronjob is set up.

#16 Updated by Nikhil Jain about 5 years ago

  • Status changed from Resolved to In Progress
  • Assignee changed from Zhengqi Yang to Michael Robson

Reassigned to Michael for upgrading bs on panache.

#17 Updated by Michael Robson about 5 years ago

Upgrade Beatufiul Soup to the newest version using: sudo pip install --upgrade beautifulsoup4 on Panahce.
Modify /etc/cron.daily/charm_manupdate to change the git repository to point to: (it needed the /gerrit/ added).
Run sudo ./charm_manupdate. Failed. Run cd /tmp/charm-manual/charm/doc, sudo make web. Failed:
Traceback (most recent call last):
File "../markupSanitizer.py", line 89, in <module>
print( soup.prettify(formatter="html") )
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0110' in position 24540: ordinal not in range(128)

Nikhil saw an error in the AMPI manual. Identify errant character using: tr '\320' '~' <manual-1p.html | grep '~'. This translates the character (first reported in hex, now in octal) to a ~ (which didn't exist in the manual originally). Find error in manual-1p.html (with Phil's help). Edit by hand. Fixed and commit submitted. Now testing if cron job actually runs.

#18 Updated by Michael Robson about 5 years ago

  • Status changed from In Progress to Resolved

Switched from building nightly using cron to using Jenkins to build on-demand whenever there is a change merged in the doc directory. I submitted a small update to ensure it worked. Merge triggered the build however the website wasn't immediately updated. A rebuild solved this issue. Not entirely sure what the problem is but it should be fine now.

Also, on an unrelated note, I tried seeing if the old cron job was still running. I can't tell (it currently still exists in /etc/cron.daily) but adding a file to cron.hourly didn't seem to have any effect. More investigation is needed, especially by a cron master.

#19 Updated by Lukasz Wesolowski about 5 years ago

This is in response to Michael's last post.

I looked at the cron configuration on panache. Cron is running as expected. For the details of how it is configured, see /etc/crontab. Note that daily, weekly, and monthly jobs are handled by anacron (/etc/anacrontab), which ensures execution of jobs with longer periods that got skipped due to the system not being up at the scheduled time.

As to the issues with the manual, errors from cron jobs are written to /var/mail/root. There, you will see that your hourly test job is being attempted by the system every hour, but that there is a problem with it (Exec format error). The manual update job is also running every day as scheduled. There is a lot of output in /var/mail/root. It was failing until very recently (June 4th). This would normally result in an email sent to sysadmin-ppl, but someone had uninstalled mail on panache, so the script was not sending the email successfully. In any case, please look at the output in /var/mail/root to check the detailed output for the manual update script. If jenkins is being used for the manual update, you can move the manual update script out of cron.daily (as well as your test job). Otherwise, install mail or change the script to use sendmail when it fails.

#20 Updated by Michael Robson about 5 years ago

  • Status changed from Resolved to Closed

I've moved the cron job charm_manupdate to root's home on Panache and deleted my hourly mail test.

In the future, we need to make sure the Jenkins job works as expected (I ran a test that seemed to have worked but I want external verification). Otherwise, this is finished.

Also available in: Atom PDF