roundup-users Mailing List for Roundup Issue Tracker (Page 3)

First, I apologize for asking about such an ancient version of the software. This is Roundup 1.5.0 with classic templates. It’s not my choice, though.

I’ve encountered a situation where Mac Safari displays a pulldown menu correctly, but Chrome and Firefox (also on Mac) do not. Chrome and Edge on Windows work correctly.

I’ve got a DB class called fbcustcomp in schema.py as a Link type, which is populated with slightly more than 100 company names. The .item.html file for this issue type has a table row:

<tr>
<th class="required" nowrap>Company</th>
<td tal:content="structure context/fbcustcomp/menu">fbcustcomp</td>
</tr>

In Mac Safari and Windows Chrome or Edge, the full list can be pulled down. In Mac Chrome or Firefox, the list is empty (“not selected”) on a new issue, and if an issue has been edited, there are exactly two items in the pulldown (“not selected” and whatever was saved in a browser where the pulldown works).

Most of the users are on Windows and Mac Safari works right, so this is not even close to an emergency. But if anyone has any thoughts about this issue, I’m all ears.

I’m also open to hearing “we haven’t got time to support a 15-year-old Roundup—get yourself current!” I’ve got it coming.

Cheers!
--
David Hancock

Hi all:

In https://jviide.iki.fi/http-redirects jviide notes that redirecting
http -> https automatically for API calls is not a great idea.

For regular human browser use, credentials aren't usualy sent with the
first http request (it waits for a 401-Unauthorized response). However
for API calls, credentials are usually supplied with the first call to
the API.

Jviide suggests returning a 403 (Forbidden) along with an explanation.
The explanation says that the credentials should be changed, since
they have been exposed in plaintext over the network.

At the very least it stops repeated successful API access over the
insecure path. This should reduce the number of times the credentials
are exposed.

It also suggests automatically voiding the credentials if a request
comes in over http, but that's a whole other issue.

Have a great day all and I hope you are enjoying Roundup 2.4.0beta2.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

On Tue, May 28, 2024 at 02:25:06PM -0400, John P. Rouillard wrote:
> Hello All:
> 
> I'm proud to release version 2.4.0 beta 2 of the Roundup issue
> tracker. This release is a bugfix and feature release, so make
> sure to read docs/upgrading.txt (
> https://www.roundup-tracker.org/dev_docs/docs/upgrading.html) to
> bring your tracker up to date.
> 
> I expect the final release to happen on July 13th.

Thanks!

Kind regards
Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: of...@ru...

Hello All:

I'm proud to release version 2.4.0 beta 2 of the Roundup issue
tracker. This release is a bugfix and feature release, so make
sure to read docs/upgrading.txt (
https://www.roundup-tracker.org/dev_docs/docs/upgrading.html) to
bring your tracker up to date.

I expect the final release to happen on July 13th.

The 67 changes, as usual, include some new features and many bug
fixes.

See details and cautions on upgrading at:

    https://www.roundup-tracker.org/dev_docs/docs/upgrading.html

Note that you should run `roundup-admin ... migrate` to update the
database schema version. Do this before you use the web, command-line
or mail interface and before any users access the tracker.

You can install it with:

   pip install roundup~=2.4.0b2

(preferably in a new virtual environment). To download it, use:

   pip download roundup~=2.4.0b2

then unpack and test/install from the tarball. Docker users can:

   docker pull rounduptracker/roundup:2.4.0b2

then run it in demo mode using the instructions at:

  https://www.roundup-tracker.org/dev_docs/docs/installation.html#running-=
in-demo-mode-with-docker

using the docker image above. For those on an ARM platform, 2.4.0b2
docker images are available at:

   rounduptracker/roundup-development:multi.

Among the notable improvements in 2.4.0 from the 2.3.0 release
are:

   * new classhelper component thanks to a team of students from
     CS682 at U-Mass Boston. This fixes many issues with the old
     classhelper. It is implemented as a web-component and needs
     REST interface access. It will fall back to the classic
     classhelper if REST is not available or if the browser does
     not support web-components.

   * fix Windows Python installation using pip. It used to go
     into an infinite loop during install or download. Also fix
     installation of shared files (templates) so roundup-admin
     can find them.

   * using @current_user as a value in a search URL for a user
     property will use the current logged in user. Now you can
     share searches like: "My issues" as "my" will become the
     current logged in user.

   * login failures to the REST/XML-RPC interfaces are now rate
     limited to limit password guessing attacks.

   * utf8mb4 is the default charset for MySQL. This requires
     migrating your database using the mysql client. You can
     choose to keep the older character set in config.ini.

   * PostgreSQL services defined in pg_service.conf can be
     used. PostgreSQL schemas are supported to eliminate the need
     for the roundup user to have database creation/deletion
     privileges.

   * fix out of memory issue when importing larger trackers into
     PostgreSQL.

   * multiple roundup-admin improvements: display protected
     properties (like creation date), better formatting of
     output, command history. Also on windows, pyreadline3 is
     supported to provide an editable interactive command line.

   * an experimental wsgi performance improvement in 2.3.0 is now
     now the default and is opt-out.

   * new template functions: utils.readfile and
     utils.expandfile. Javascript that is included in the Python
     core will be moved to external files and be able to have
     values from Roundup substituted in the Javascript.

   * allow content-type of a template to be set from inside the
     template. This allows returning json or xml from a template
     without a .json or .xml extention.

   * fix import/export on windows to use Unix style line endings
     fixing export/import on Windows and making exports portable
     across platforms.

   * various other Windows platform fixes including test suite
     fixes.

   * sqlite version 1 and StructuredText support removed.

The file CHANGES.txt has a detailed list of feature additions and
bug fixes (67) for each release. The most recent changes from
there are at the end of this announcement. Also see the
information in doc/upgrading.txt.

If you find bugs, please report them to issues AT
roundup-tracker.org or create an account at
https://issues.roundup-tracker.org and open a new ticket. If you
have patches to fix the issues they can be attached to the email
or uploaded to the tracker.

Upgrading
=========

If you're upgrading from an older version of Roundup you must
follow the Software Upgrade guidelines given in the
doc/upgrading.txt documentation. Online at:

   https://www.roundup-tracker.org/dev_docs/docs/upgrading.html

Note that you should backup your database and run `roundup-admin
... migrate` for all your trackers to update the database schema
version. Do this before you use the web, command-line or mail
interface and before any users access the tracker.

Roundup requires Python 2 newer than version 2.7.12 or Python 3
newer than or equal to version 3.6 for correct operation. (Python
3.4 or 3.5 may work, but are not tested.) Note that Python 2
support is being removed from the CI platforms, so you should
deploy new trackers with Python 3 and plan on upgrading older
trackers from Python 2 to Python 3. See the upgrade guide.

To give Roundup a try, just download (directions above), unpack
and run:

   python demo.py

then open the url printed by the demo app.

Release info and download page:

    https://pypi.org/project/roundup/2.4.0b2/

Source and documentation is available at the website:

    https://www.roundup-tracker.org/dev_docs/

Mailing lists - the place to ask questions:

    https://sourceforge.net/p/roundup/mailman/

About Roundup
=============

Roundup is a simple-to-use and install issue-tracking system with
command-line, web and e-mail interfaces. It is based on the
winning design from Ka-Ping Yee in the Software Carpentry "Track"
design competition.

Roundup manages a number of issues (with flexible properties such
as "description", "priority", and so on) and provides the ability
to:

(a) submit new issues,
(b) find and edit existing issues, and
(c) discuss issues with other participants.

The system facilitates communication among the participants by
managing discussions and notifying interested parties when issues
are edited. One of the major design goals for Roundup that it be
simple to get going. Roundup is therefore usable "out of the box"
with any Python 3.6+ installation. It doesn't even need to be
"installed" to be operational, though an install script is
provided.

It comes with five basic issue tracker templates

* a classic bug/feature tracker
* a more extensive devel tracker for bug/features etc.
* a responsive version of the devel tracker
* a jinja2 version of the devel template (work in progress)
* a minimal skeleton

and supports four database back-ends (anydbm, sqlite, mysql and
postgresql).

Recent Changes
==============

>From 2.3.0 to 2.4.0

TBA

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Hi all:

Sourceforge's mailing lists have been broken for most of the weekend.

Before I send out an announcement for 2.4.0b2 release, I want to make
sure this goes through first.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Hi David:

In message <5F3...@po...>, David
Hancock writes:
>Thank you! That all makes sense. I am using an iptables rule to
>allow 8080 from anywhere. I’d better fix that pronto.

More restrictive firewall rule for the win!

>Understood about roundup-server vs. more robust chains. For the
>moment I’m just trying to relearn schema and template changes. Those
>will get rolled into a proper Roundup, behind Apache with https and
>better firewalls.
>
>The super-convenient roundup-server is handy, but not so great if I’m
>open to attacks!

Agreed. I have been working on Roundup with some graduate student
teams this past semester.  Being able to spin up a server quickly for
development/testing out ideas has been a major win.

Have a great week.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Thank you! That all makes sense. I am using an iptables rule to allow 8080 from anywhere. I’d better fix that pronto. 

Understood about roundup-server vs. more robust chains. For the moment I’m just trying to relearn schema and template changes. Those will get rolled into a proper Roundup, behind Apache with https and better firewalls. 

The super-convenient roundup-server is handy, but not so great if I’m open to attacks!

Cheers!
--
David Hancock (sent from my phone)

On May 18, 2024, at 18:19, John P. Rouillard <ro...@cs...> wrote:

Hi David:

In message <F6E...@po...>,
David Hancock writes:
> Roundup 2.3.0 source distribution, Python 3.10.12, running
> roundup-server with http on port 8080, net 0.0.0.0. I have
> SENDMAILDEBUG writing to a local file.

Are you running a firewall on your host and is your host connected to
the public internet?

> I’d like to understand the errors that follow. Here’s the server
> running in the foreground, followed by a request from my IP for the
> list of issues, followed by two pairs of error messages from IP
> addresses that I don’t recognize.

Details below, but this looks like you are being scanned/probed.

> Any ideas what these are (or pointers for finding this myself)? Here
> is what I’m seeing:
> 
> (simona-roundup) dhancock@ubuntu-s-1vcpu-1gb-nyc3-01:~/trackers/simona$ roundup-server -p 8080 -n 0.0.0.0 simona=/home/dhancock/trackers/simona
> 
> Roundup server started on 0.0.0.0:8080
> 
> 100.16.88.22 - - [18/May/2024 21:00:25] "GET /simona/issue?@sort=-activity&@group=priority&@filter=status&@columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignedto%2Cstatus&@search_text=&status=-1%2C1%2C2%2C3%2C4%2C5%2C6%2C7&@dispname=Show%20All&@pagesize=50&@startwith=0 HTTP/1.1" 200 -
> 
> 149.56.23.153 - - [18/May/2024 21:00:30] code 400, message Bad request syntax ('\\x16\\x03\\x01\\x02\\x00\\x01\\x00\\x01ü\\x03\\x03\\x94OW')
> 
> 149.56.23.153 - - [18/May/2024 21:00:30] "\x16\x03\x01\x02\x00\x01\x00\x01ü\x03\x03\x94OW" 400 -

I am running a dev version of Roundup on http. If I hit the endpoint
using HTTPS not HTTP, I see:

192.168.1.20 - - [18/May/2024 17:45:02] code 400, message Bad request version ("y$¯OÞU\\x00F\\x13\\x02\\x13\\x03\\x13\\x01À,À0̨̩À\\xadÀ+À/À¬À#À'À")
192.168.1.20 - - [18/May/2024 17:45:02] "\x16\x03\x01\x02\x00\x01\x00\x01ü\x03\x03A\x93#6ª..." 400 -

(... is elided) which looks a lot like your output.

See also: https://stackoverflow.com/questions/60144556/apache-httpd-server-doesnt-accept-get-http-request-from-f5-load-balancer

> 
> 87.121.69.52 - - [18/May/2024 21:02:09] code 501, message Unsupported method ('CONNECT')
> 
> 87.121.69.52 - - [18/May/2024 21:02:09] "CONNECT google.com:443 HTTP/1.1" 501 -

This is a connect/tunnel request from some system/scanner on the
internet. https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT

It looks like your test box is on the open internet and is being hit
by random scanners/hackers.  My suggestion is to enable a host
firewall and block all IP connections to port 8080 that are not from
the hosts you are using to test the server (network 100.16.88/24
maybe). If your server and test client are on a local lan, use '-n
<server local lan address>' and configure config.ini to use the local
lan address.

Roundup-server is fine for production, but it doesn't have the
features/performance required to defend against an attack. If you are
allowing access from the internet, we recommend placing roundup-server
behind a reverse proxy. Reverse proxies (e.g. Apache, varnish, nginx,
Hiawatha ...) can run web application firewalls (WAF), rate limit or
filter connections. Roundup-server takes longer to process/drop a
connection and doesn't have WAF and other filtering capabilities.

Using roundup-server directly on a local lan/intranet protected from
the internet by a firewall is fine as you don't expect people to
actively attack the server to be on your local lan.

> This seems to happen only on the first request after starting the
> roundup server, and then they stop. And I’ve just observed that after
> subsequent restarts, they don’t appear for a few real requests, then
> I get a few more of the code 400s.

Timing of the connections could be a scanning/hacking tool that hits
an endpoint until it gets an answer. Then slows or stops probes. Your
'first request' observation could be regular probing of your system
that is discarded because nothing is listening. Then when you fire up
Roundup, well it's listening and you see the probe that was invisible
before.

Most hosts have host level firewalls. You might see more activity of
probing by looking at your firewall logs.

Hope this helps.
--
               -- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Hi David:

In message <F6E...@po...>,
David Hancock writes:
>Roundup 2.3.0 source distribution, Python 3.10.12, running
>roundup-server with http on port 8080, net 0.0.0.0. I have
>SENDMAILDEBUG writing to a local file.

Are you running a firewall on your host and is your host connected to
the public internet?

>I’d like to understand the errors that follow. Here’s the server
>running in the foreground, followed by a request from my IP for the
>list of issues, followed by two pairs of error messages from IP
>addresses that I don’t recognize.

Details below, but this looks like you are being scanned/probed.

>Any ideas what these are (or pointers for finding this myself)? Here
>is what I’m seeing:
>
>(simona-roundup) dhancock@ubuntu-s-1vcpu-1gb-nyc3-01:~/trackers/simona$ roundup-server -p 8080 -n 0.0.0.0 simona=/home/dhancock/trackers/simona
>
>Roundup server started on 0.0.0.0:8080
>
>100.16.88.22 - - [18/May/2024 21:00:25] "GET /simona/issue?@sort=-activity&@group=priority&@filter=status&@columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignedto%2Cstatus&@search_text=&status=-1%2C1%2C2%2C3%2C4%2C5%2C6%2C7&@dispname=Show%20All&@pagesize=50&@startwith=0 HTTP/1.1" 200 -
>
>149.56.23.153 - - [18/May/2024 21:00:30] code 400, message Bad request syntax ('\\x16\\x03\\x01\\x02\\x00\\x01\\x00\\x01ü\\x03\\x03\\x94OW')
>
>149.56.23.153 - - [18/May/2024 21:00:30] "\x16\x03\x01\x02\x00\x01\x00\x01ü\x03\x03\x94OW" 400 -

I am running a dev version of Roundup on http. If I hit the endpoint
using HTTPS not HTTP, I see:

 192.168.1.20 - - [18/May/2024 17:45:02] code 400, message Bad request version ("y$¯OÞU\\x00F\\x13\\x02\\x13\\x03\\x13\\x01À,À0̨̩À\\xadÀ+À/À¬À#À'À")
 192.168.1.20 - - [18/May/2024 17:45:02] "\x16\x03\x01\x02\x00\x01\x00\x01ü\x03\x03A\x93#6ª..." 400 -

(... is elided) which looks a lot like your output.

See also: https://stackoverflow.com/questions/60144556/apache-httpd-server-doesnt-accept-get-http-request-from-f5-load-balancer

>
>87.121.69.52 - - [18/May/2024 21:02:09] code 501, message Unsupported method ('CONNECT')
>
>87.121.69.52 - - [18/May/2024 21:02:09] "CONNECT google.com:443 HTTP/1.1" 501 -

This is a connect/tunnel request from some system/scanner on the
internet. https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT

It looks like your test box is on the open internet and is being hit
by random scanners/hackers.  My suggestion is to enable a host
firewall and block all IP connections to port 8080 that are not from
the hosts you are using to test the server (network 100.16.88/24
maybe). If your server and test client are on a local lan, use '-n
<server local lan address>' and configure config.ini to use the local
lan address.

Roundup-server is fine for production, but it doesn't have the
features/performance required to defend against an attack. If you are
allowing access from the internet, we recommend placing roundup-server
behind a reverse proxy. Reverse proxies (e.g. Apache, varnish, nginx,
Hiawatha ...) can run web application firewalls (WAF), rate limit or
filter connections. Roundup-server takes longer to process/drop a
connection and doesn't have WAF and other filtering capabilities.

Using roundup-server directly on a local lan/intranet protected from
the internet by a firewall is fine as you don't expect people to
actively attack the server to be on your local lan.

>This seems to happen only on the first request after starting the
>roundup server, and then they stop. And I’ve just observed that after
>subsequent restarts, they don’t appear for a few real requests, then
>I get a few more of the code 400s.

Timing of the connections could be a scanning/hacking tool that hits
an endpoint until it gets an answer. Then slows or stops probes. Your
'first request' observation could be regular probing of your system
that is discarded because nothing is listening. Then when you fire up
Roundup, well it's listening and you see the probe that was invisible
before.

Most hosts have host level firewalls. You might see more activity of
probing by looking at your firewall logs.

Hope this helps.
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Roundup 2.3.0 source distribution, Python 3.10.12, running roundup-server with http on port 8080, net 0.0.0.0. I have SENDMAILDEBUG writing to a local file.

Background: I haven’t done any Roundup work for at least 6 years; that server is Python 2.7 and a much older Roundup. I’m prototyping some additions (a new issue type), and I’m running a standalone server for the moment. 

I’d like to understand the errors that follow. Here’s the server running in the foreground, followed by a request from my IP for the list of issues, followed by two pairs of error messages from IP addresses that I don’t recognize. This seems to happen only on the first request after starting the roundup server, and then they stop. And I’ve just observed that after subsequent restarts, they don’t appear for a few real requests, then I get a few more of the code 400s.

Any ideas what these are (or pointers for finding this myself)? Here is what I’m seeing:

(simona-roundup) dhancock@ubuntu-s-1vcpu-1gb-nyc3-01:~/trackers/simona$ roundup-server -p 8080 -n 0.0.0.0 simona=/home/dhancock/trackers/simona

Roundup server started on 0.0.0.0:8080

100.16.88.22 - - [18/May/2024 21:00:25] "GET /simona/issue?@sort=-activity&@group=priority&@filter=status&@columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignedto%2Cstatus&@search_text=&status=-1%2C1%2C2%2C3%2C4%2C5%2C6%2C7&@dispname=Show%20All&@pagesize=50&@startwith=0 HTTP/1.1" 200 -

149.56.23.153 - - [18/May/2024 21:00:30] code 400, message Bad request syntax ('\\x16\\x03\\x01\\x02\\x00\\x01\\x00\\x01ü\\x03\\x03\\x94OW')

149.56.23.153 - - [18/May/2024 21:00:30] "\x16\x03\x01\x02\x00\x01\x00\x01ü\x03\x03\x94OW" 400 -

87.121.69.52 - - [18/May/2024 21:02:09] code 501, message Unsupported method ('CONNECT')

87.121.69.52 - - [18/May/2024 21:02:09] "CONNECT google.com:443 HTTP/1.1" 501 -

Thank you for any help you can provide.

P.S. I was also going to report that the Roundup Wiki was down, but it’s back now.

Cheers!
-- 
David Hancock

Hi everybody:

In preparation for release 2.4 I have spent some time fixing tests
(and code) on the windows platform.

If you run Roundup on windows, can you download the following zip file,

  https://github.com/roundup-tracker/roundup/archive/f6d473d27689ddcaf68bf1a2801045834ea3c8d9.zip

unpack it and run:

  python -m pytest test

and let me know if you pas everything or fail any tests. This tests
just the basics using stdlib with pytest (and optionally requests).  I
don't have MySQL/PostgreSQL/xapian/.. installed on the windows box I
have access to so only the basic out of the box stuff works.

(Note: I am sharing GitHub's download URL because Sourceforge's links
for downloading snapshots become invalid after a time.)

Thanks.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Hi Stefan/Ralf:

I must have missed this email originally (hmm, given the timestamp I
may not have received it yet). I read it on the ML archive.

In message <202...@ru...>,
Ralf Schlatterbeck writes:
>On Mon, Apr 29, 2024 at 11:31:43AM +0000, Schmelz, Stefan wrote:
>[...]
>> And this ist he HTML response in My browser:
>> Error response
>> Error code: 404
>> Message: /issue2/?@ok_message=Der%20Eintrag%20%22msg3%22%20wurde%20erstellt%0ADer%20Eintrag%20%22issue2%22%20wurde%20erstellt&@template=item.
>> Error code explanation: 404 - Nothing matches the given URI.

I'm sorry you are running into problems. This is an odd one.

>Are you running this with the standalone server or behind a web server
>like apache or nginx?

That is my first question as well. If the URL is really:

   .../issue2/?...

that's wrong. It should be:

  .../issue2?...

(note no '/' before the query part). I can confirm that:

    https://myhost/demo/issue157

displays issue 157 while:

    https://myhost/demo/issue157/

reports a 404. So I can reproduce your results, but not your cause.

>Let us know if you make progress on this!

I agree. It looks like something is messing up the URL.

URL's that point to files (e.g. /file23) use a trailing '/' and
optional name component to trigger a download of the file
contents. E.G. /file23/myfile would download the contents of file23
and suggest the name "myfile" in the file save dialog. '/file23/'
would start the download with the file name "download" suggested in the
dialog.

However other object types don't support the trailing '/' in the URL
and return 404. If Roundup is really returning the trailing /, it's a
bug that I have never seen. To help us help you can you supply:

  Version of Roundup (roundup-admin -v)

  Version of Python (python -V)

  Operating system

  How you installed Roundup ('pip install roundup', download the
      tarball/release and ran setup.py ...)

  How you are accessing Roundup. Via roundup-server? WSGI with
      gunicorn etc. Is there a cache server or reverse proxy involved?

Thanks and sorry your having a tough time here.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Hi Stefan,

On Mon, Apr 29, 2024 at 11:31:43AM +0000, Schmelz, Stefan wrote:
[...]
> And this ist he HTML response in My browser:
> Error response
> Error code: 404
> Message: /issue2/?@ok_message=Der%20Eintrag%20%22msg3%22%20wurde%20erstellt%0ADer%20Eintrag%20%22issue2%22%20wurde%20erstellt&@template=item.
> Error code explanation: 404 - Nothing matches the given URI.

Are you running this with the standalone server or behind a web server
like apache or nginx?

What have you configured in config.ini in section [tracker], config
variable 'web'? This must match exactly what you have configured in the
web server or what you have specified on the command-line of
roundup-server. The argument passed to roundup-server is the name of the
first URL component after the hostname -- that would go *before* the
"/issue2" in your Message above. This must match what you've configured
in 'web'.

So as 'web' you would have something like
web = http://hostname/trackername/
and you would call roundup-server with
roundup-server trackername=/path/to/tracker/

Note that the 'web' config variable needs a trailing '/'.

Let us know if you make progress on this!

Thanks
Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: of...@ru...

Hi There,
I just installed roundup as a test and i have to report that it does not work that well so far.
If I create, or update anything, there is a success message and that yields a 404 Error every time.

This ist he Server Log if I create a new Issue that is assigned to me:
192.168.24.182 - - [29/Apr/2024 11:25:40] "GET /issue2/?@ok_message=Der%20Eintrag%20%22msg3%22%20wurde%20erstellt%0ADer%20Eintrag%20%22issue2%22%20wurde%20erstellt&@template=item HTTP/1.1" 404 -

And this ist he HTML response in My browser:
Error response
Error code: 404
Message: /issue2/?@ok_message=Der%20Eintrag%20%22msg3%22%20wurde%20erstellt%0ADer%20Eintrag%20%22issue2%22%20wurde%20erstellt&@template=item.
Error code explanation: 404 - Nothing matches the given URI.

I don't know what to do and what could cause it. I did some digging in the config, but that was inconclusive and the mailing list so far did not yield any useful answers either...

Pls Help, I#m starting to feel retarded.
Regards
Stefan


--
Solupharm Pharmazeutische Erzeugnisse GmbH
Industriestra?e 3, D-34212 Melsungen
Tel: +49 (5661) 7305-767
Email: Ste...@so...
Homepage: https://www.solupharm.com


[lslp_ogo] [Vision_mission]


Folgen Sie uns auf Facebook! [https://mail.solupharm.com/owa/auth/15.1.2375/themes/resources/facebook.png] <https://www.facebook.com/Solupharm-Pharmazeutische-Erzeugnisse-GmbH-100921692587632/>

-----------------------------------------------------------------------------------------------
Gesch?ftsf?hrer: Friedemann Seitz, Thomas Zimmermann, Dr. J?rgen G?b
Sitz der Gesellschaft: D-34212 Melsungen
HRB 11245, Amtsgericht Fritzlar, USt-ID DE-203368511
-----------------------------------------------------------------------------------------------

Hi Everybody:

MySQL has had a checkered past with supporting UTF8 charsets.  Roundup
has shared that challenge. The utf8mb4 preferred by Roundup since
2022, has some issues. This includes the crashes reported by Nagy
Gabor when Roundup mixes utf8mb4 charsets with incompatible
collations. This latest patch hopefully fixes all of those and chooses
the best defaults while letting you fall back to older charsets.

It also include instructions on converting the underlying database
from whatever encoding/collation to a
utf8mb4/utf8_unicode_ci/utf8_0900_bin combination.

I have done some minimal testing, but nothing I can do would match a
conversion on a real live tracker.

So I am asking those of you with trackers running on MySQL to look at
the commit including documentation at:

  https://sourceforge.net/p/roundup/code/ci/8b31893f5930cbea37031d49bd08a631d1b6b654/

and test this. It will be part of the 2.4.0 release in July. I plan on
putting out a beta release (installable by pip) in mid May, but having
this tested before then would really help.

Thanks and have a great week.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

On 4/3/24 14:38, John P. Rouillard wrote:

> and I think that's it. The ownership on aliases.roundup should cause
> roundup-mailgw to run as the roundup user according to local(8). The
> (writable) tracker files are owned by the roundup_user.
> 
> I think all the trackers are owned by the same roundup user. If not, a
> separate alias file for each user would need to be created.
> 
> Thoughts?

I think that's a nice approach. I inherited our tracker, so everything 
was already set up once I took over.

This is our approach. It uses two servers. One is an internet-facing 
postfix server that routes incoming emails to another server that is 
hosting roundup. It uses procmail and local postfix installation as MTA.

/etc/postfix(virtual-aliases on internet facing postfix.

The internet facing Postfix server performs spam checks using 
Spamassassin, and set's a header, X-Spam-Status:

/etc/postfix(virtual-aliases on internet facing postfix.

su...@my...               su...@tr...
dem...@my...           dem...@tr...

/etc/aliases of internal machine postfix (tracker.mydomain.se)
support:        roundup+support
demosupport:    roundup+demosupport

Content of $HOME/.procmailrc+support of roundup user:
LOGFILE=$HOME/.procmail_support.log

:0
* ^X-Spam-Status: Yes
spam/support/

:0
|/usr/local/bin/roundup-mailgw -S external=True 
/opt/roundup/trackers/support-tracker/

And $HOME/.procmailrc+demosupport of roundup user:

LOGFILE=.procmail_demosupport.log

:0
* ^X-Spam-Status: Yes
spam/demosupport/

:0
|/usr/local/bin/roundup-mailgw -C issue -S 'priority=Demosupport' 
/opt/roundup/trackers/support-tracker/
#! dem...@my...

As you can see, this can easily be extended to other tracker instances 
as needed. The demosupport@ will also trigger a notification to a Slack 
channel, as well as deliver the message to a seperate Tracker queue

Kind regards,
-- 
Martin Östlund          Support
Cendio AB               https://cendio.com
Teknikringen 8          https://twitter.com/ThinLinc
583 30 Linköping        https://facebook.com/ThinLinc
Community Support       https://community.thinlinc.com

Hi Martin:

In message <078...@ce...>,
=?UTF-8?Q?Martin_=C3=96stlund?= via Roundup-users writes:
>On 4/2/24 23:42, John P. Rouillard wrote:
>> I am trying to help somebody setup inbound email using postfix for
>> multiple roundup trackers on two different virtual addresses. For this
>> example: is...@ex... and is...@an....
>> 
>> Does anybody have a setup that does this and runs the roundup-mailgw
>> with the proper uid/gid?
>
>Not with multiple trackers, but with a single tracker instance. If I 
>were to have multiple trackers I would probably make Postfix route
>is...@ex... and is...@an... to the appropiate tracker.

I know that is a possibility, although I have never used it. Do you
have an example of this?

>What issues are you seeing?

The system got upgraded and apparently the postfix config got lost
during the upgrade.

I think this will work, but something simpler would be good as well.

The box is known as abox.com on the internet but receives email for:
is...@ex... and bu...@an....

in /etc/postfix/main.cf add:

  virtual_alias_domains = example.com another.com
  virtual_alias_maps = hash:/etc/postfix/virtual

  alias_maps = hash:/etc/aliases hash:/etc/postfix/aliases.roundup

in /etc/postfix/virtual add:

  # after changing this run 'postmap /etc/postfix/virtual'

  is...@ex...    roundup_issues_example
  bu...@an...      roundup_bugs_another

Create /etc/postfix/aliases.roundup with owner/group
roundup_user/roundup_group. Add:

  # after changing this run 'postmap /etc/postfix/aliases.roundup'

  roundup_issues_example |"/tools/roundup/bin/roundup-mailgw /d/track/example"
  roundup_issues_another |"/tools/roundup/bin/roundup-mailgw /d/track/another"

and I think that's it. The ownership on aliases.roundup should cause
roundup-mailgw to run as the roundup user according to local(8). The
(writable) tracker files are owned by the roundup_user.

I think all the trackers are owned by the same roundup user. If not, a
separate alias file for each user would need to be created.

Thoughts?
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

On 4/2/24 23:42, John P. Rouillard wrote:
> Hi Folks:

Hello!

> I am trying to help somebody setup inbound email using postfix for
> multiple roundup trackers on two different virtual addresses. For this
> example: is...@ex... and is...@an....
> 
> Does anybody have a setup that does this and runs the roundup-mailgw
> with the proper uid/gid?

Not with multiple trackers, but with a single tracker instance. If I 
were to have multiple trackers I would probably make Postfix route
is...@ex... and is...@an... to the appropiate tracker.

What issues are you seeing?

Kind regards,
--
Martin Östlund          Support
Cendio AB               https://cendio.com
Teknikringen 8          https://twitter.com/ThinLinc
583 30 Linköping        https://facebook.com/ThinLinc
Community Support       https://community.thinlinc.com

On Tue, Apr 02, 2024 at 05:42:01PM -0400, John P. Rouillard wrote:
> Hi Folks:
> 
> I am trying to help somebody setup inbound email using postfix for
> multiple roundup trackers on two different virtual addresses. For this
> example: is...@ex... and is...@an....
> 
> Does anybody have a setup that does this and runs the roundup-mailgw
> with the proper uid/gid?
> 
> Also I would like to document this better in the installtion
> document. Thanks.

I've generally moved to assigning an imap (or pop3) mailbox to roundup
and use polling with the imap scripts: This avoids that during
maintenance or other downtime mail might vanish (because at the time the
mail is tried to deliver to the mailbox the mailserver has already
accepted the mail) and it avoids all the permission problems.
In addition you don't need a mailserver on the roundup box.

So one reason that it's not well documented is that in todays
environment it is not in much use...

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: of...@ru...

Hi Folks:

I am trying to help somebody setup inbound email using postfix for
multiple roundup trackers on two different virtual addresses. For this
example: is...@ex... and is...@an....

Does anybody have a setup that does this and runs the roundup-mailgw
with the proper uid/gid?

Also I would like to document this better in the installtion
document. Thanks.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Hi all:

A couple of emails I sent to the list bounced after a few days. Just
trying to see if things are still broken.

--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.

Hi all:

It appears that my main email site can't send anything to sourceforge lists.
sorry if this becomes a repeat.

Hi r2d2:

In message <414...@mi...>,
r2...@mi... writes:
>I would like to restrict the unlinking (and relinking) of issues /
>messages / files for "user" roles and only allow it for "admin"
>roles.

Adding/removing an item from the messages multilink is audited. So you
will have a record of what happened in any case. Setting the whole
messages multilink will be recorded as well but I am not sure if the
history shows multiple add/remove or a single set in the history. Ralf
any idea?

>Is this possible with some access control rules,

Unlinking can be stopped by an auditor. The idea is (untested and
probably uninterpretable Python like code below):

    audit(db, cl, itemid, newdata)

       # Is the user changing messages
       if 'messages' not in newdata: return

       # Is the user removing or replacing a message?
       current_messages = cl.get(itemid, 'messages')
       new_messages = newdata['messages']
       removed = [message for message in current_messages if message
                  not in new_messages]

       if not removed: return

       # If the user is an Admin hasPermission always passes, but
       # let's say we wanted to add a user who can edit messages
       # if they have the moderator permission.
       #
       # See: https://roundup-tracker.org/docs/customizing.html#restricting-the-list-of-users-that-are-assignable-to-a-task

       if not db.security.hasPermission('moderator', userid, cl.classname):
        raise ValueError('User %s does not have moderator role, '
                         'and can not remove messages from an issue')

Note that this code doesn't quite do what you think it would:

       if len(newdata['messages'] > len(cl.get(itemid, 'messages')):
          # user made a comment and is adding to the messages (NOT!)
          return

While @remove can only remove, not replace, items in the messages list,
a set/edit operation can. So I could remove item id 10 and replace it
with item id 55 in one transaction. This preserves the length of the
list. This could be done using the Rest interface as well. If I am
wrong, somebody can correct me.

Detecting re-adding is a bit trickier. Adding (by commenting) and
re-adding look kind of identical from the auditor's viewpoint.  There
is no database method to report if a message id was previously assigned
to the issue.  However for new messages/comments the message id being
added will either:

     be missing from the db as the transaction to create the msg and
       update the db isn't committed yet. So

         cl.db.msg(added_message_id, 'creation')

       will raise an exception. I think this is the correct case.

     have its creation date within a second or two of now() and the
       creator will be the current user.

To detect re-adding, you can create a "removed_messages" property and
look to see if your added message:

  added = [message for message in new_messages if message
            not in current_messages]

is in the "removed_messages" property. An auditor would have to keep
the removed_messages property list up to date.

Another way is to grovel through the issue's history and look for
entries that remove/unlink or set (possibly replacing one or more
message ids) the messages property. You would start from the
current values of messages and walk backward through history recording
any unlinked items and calculating any removed items when a set
operation happens.

Can it be done, yes. Is checking for re-adding worth it? Probably not.

If a user tries to add a currently existing message, it won't do
anything as you can't have duplicates in the messages list.

>or should I just purge the "remove" button from the html template?

Removing the button will stop people from accidentally unlinking a
message.  But they can still unlink the message by sending the correct
URL request to the server. Depending on your threat profile, removing
the button could be fine.

You can show the "remove" button only to Admin users. This bit of TAL
(edited from one of my trackers) shows the remove form/button only if
the user has the Admin role.

 <form method="POST"  tal:condition="python:request.user.hasRole('Admin')""
     tal:attributes="action string:issue${context/id}">
   <input type="hidden" name="@remove@messages" tal:attributes="value msg/id">
   <input type="hidden" name="@action" value="edit">
   <input name="@csrf" type="hidden" tal:attributes="value
       python:utils.anti_csrf_nonce()">
   <input type="submit" name="remove" value="remove" i18n:attributes="value">
 </form>

Hope this helps.

-- rouilj

Hi r2d2:

I answered on IRC, but I can condense/fix and copy it here. From IRC:

> so here is a link to the codeline https://codeberg.org/R2D2/roundup_todo_tracker_template/src/commit/e9eb1610830f8c188de1bfa0c6fbd8d560739d84/todo/html/issue.item.html#L79
>11:04.55 (EDT) - r2d2: i want to extend this with a second keyword field named project
>11:05.57 (EDT) - r2d2: is there a way to use something like an format-string within the filter='...' expression?

You can use printf formatting:

      filter='status=waiting,in progress,needs attention;keyword=%s' %
context.keyword.plain()

it's just python at that point. I assume you are trying to find all
"waiting,in progress" status  ... issues that match the
current context. Searching with mutlilinks is a bit tricky. What you
have I think will be an or search by default.

So if there were two keywords you would get any project that has either keyword.

You have to create a boolean expression if you have a multilink. If
you have two keywords you would filter using keyword1,keyword2,-3
which is an RPN/postfix expression and'ing (-3) the two keywords. See
the definition of filter in
roundup/hyperdb.py.

If you are going to be doing complex filtering, add a utility function
and do the work in the utility function. Trying to do any extensive
logic/python programming within the template is unreadable/subject to
quoting issues (as you kind of lose the " or need to escape it since
its used to delimit the tal expression.

See https://roundup-tracker.org/docs/customizing.html#adding-a-time-log-to-your-issues-4
for an example of registering a utility method.

-- rouilj


On Sun, Mar 10, 2024 at 1:49 PM <r2...@mi...> wrote:
>
> Hello Everyone,
>
> I struggle to understand a line in issue.item.html (based on the classic tracker template).
>
> I duplicated everything for keywords (renamed to projects), but the filter statement eludes me:
> - here is a link to the codeline https://codeberg.org/R2D2/roundup_todo_tracker_template/src/commit/e9eb1610830f8c188de1bfa0c6fbd8d560739d84/todo/html/issue.item.html#L79
> - I want to extend this with a second keyword field named project
> - Is there a way to use something like an format-string within the filter='...' expression?
>
>
> Thanks in advance!
>
>
> --
> Have a fine week!
>
> r2d2
>
>
> _______________________________________________
> Roundup-users mailing list
> Rou...@li...
> https://lists.sourceforge.net/lists/listinfo/roundup-users

Hello Everyone,

I struggle to understand a line in issue.item.html (based on the classic tracker template).

I duplicated everything for keywords (renamed to projects), but the filter statement eludes me:
- here is a link to the codeline https://codeberg.org/R2D2/roundup_todo_tracker_template/src/commit/e9eb1610830f8c188de1bfa0c6fbd8d560739d84/todo/html/issue.item.html#L79
- I want to extend this with a second keyword field named project
- Is there a way to use something like an format-string within the filter='...' expression?


Thanks in advance!


--
Have a fine week!

r2d2

Hello Everyone,

I would like to restrict the unlinking (and relinking) of issues / messages / files
for "user" roles and only allow it for "admin" roles.

Is this possible with some access control rules, or should I just purge the "remove"
button from the html template?


Thanks in advance!


--
Have a nice day!

r2d2

Hi R2d2:

In message <139...@mi...>,
r2...@mi... writes:

>I found
>https://www.roundup-tracker.org/docs/developers.html#internationalization-notes

Let me say first internationalization (i18n from now on) is not my
strong suit. I'm no C-3PO.

>But for me it is not clear if and how a new / external tracker
>template, like the todo template
>https://codeberg.org/R2D2/roundup_todo_tracker_template , fits into
>this translation system.

The deployed tracker can have a locale/ru.po, locale/de.po ... with
additional strings. IIRC this is searched first by the i18n code. If
no match is found for the key string, the installed locale files are
used.

>If I copy the todo tracker template into the roundup source tree (
>venv/share/roundup/templates ) the extraction and translation process
>works as described.  But how would this work if the todo tracker is
>not integrated into roundup?  Could an additional catalog be created
>and used in extension to the standard translation?

That is my understanding. See above.

>Since most of the translation strings of the todo tracker are already
>in the classic tracker.

You would only need to add the new strings using roundup-gettext to
create the pot file and then you can create .po files from that and
place them in the locale sub-directory of the installed tracker. IIUC
this is meant to allow translating stuff like status names which are
able to be changed on a per tracker basis.

AFAIK, you can publish your tracker template with a locale
subdirectory. The 'roundup-admin install' command just recursively
copies the template directory. This should copy any locale/* files as
well into the deployed tracker home.

>On a sidenote, did you know of Babel (pybabel)
>https://babel.pocoo.org/en/latest/ ?  Babel is an integrated
>collection of utilities that assist in internationalizing and
>localizing Python applications, with an emphasis on web-based
>applications.
>
>Babel provides tools to build and work with gettext message catalogs
>Babel includes a command-line interface for working with message
>catalogs, similar to the various GNU gettext tools commonly available
>on Linux/Unix systems.

As a replacement for xgettext, it looked workable. A module could
probably be built to extract i18n text from TAL or Jinja templates as
well.

How to handle extraction of deferred translations (including
doc strings which supply the built in help for roundup-admin) is an
open question. It would be nice to replace xpot which is old and
unmaintained.

As far as replacing/augmenting the Python native gettext, that would
have to be done with a far better grasp of i18n/i10n than I have. It
would also make roundup internationalization dependent on an external
library which is not really a good thing.

Have a great day.
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.