17747
Reporter: feedback bot
Assignee: fmendez
Type: Feedback
Summary: Rate limiting not handled by the portal-web
Priority: Unassessed
Resolution: Duplicate
Status: Closed
Created: 2015-08-05 15:18:46.348
Updated: 2017-10-10 11:00:46.878
Resolved: 2017-10-10 11:00:46.857
Description: I was on this type of page : " http://www.gbif.org/occurrence/search?TAXON_KEY=Turdus+merula&HAS_COORDINATE=true&HAS_GEOSPATIAL_ISSUE=false&GEOMETRY=21.09375+72.607120%2C-19.6875+58.813741%2C-15.468749+34.307143%2C43.59375+43.068887%2C46.40625+68.138851%2C21.09375+72.607120&YEAR=2001%2C2015&MONTH=10%2C12&MONTH=1%2C5#
I clicked on the uper-right button "download", I chose csv format and then this message appeared :"Oops ! It seems you have discovered an unforseen error!
Please use the Feedback button on the right to file an issue.Thank you for helping to improve the service."
Can you help me ?
Thank you.
*Reporter*: florent hawotte
*E-mail*: [mailto:Florent.Hawotte@uclouvain.be]]]>
Author: ahahn@gbif.org
Created: 2015-08-06 09:40:16.687
Updated: 2015-08-06 09:40:16.687
The reported issue does no longer show up. However, the download does not finish, either, with the portal page reporting status "preparing" hours later. The same effect could be reproduced a day later.
[~fmendez@gbif.org] investigated and found that the download does finish, but the status is not getting updated in the registry database. This means that the user never gets an opportunity to access the downloaded data.
Author: ahahn@gbif.org
Comment: On retry 10 minutes later, the described "Oops" error is back.
Created: 2015-08-06 09:53:15.629
Updated: 2015-08-06 09:53:15.629
Author: trobertson@gbif.org
Created: 2015-08-06 10:31:01.484
Updated: 2015-08-06 10:36:35.023
This happens when the rate limiting kicks in. Downloads are limited depending on how much load there is on the server and how many downloads the user has running.
The portal-web logs show:
{code}
ERROR [2015-08-06 10:29:18,728+0200] [http-bio-8080-exec-5102] com.opensymphony.xwork2.interceptor.ExceptionMappingInterceptor: HTTP 420:
com.sun.jersey.api.client.UniformInterfaceException: HTTP 420:
at org.gbif.ws.client.interceptor.HttpErrorResponseInterceptor.invoke(HttpErrorResponseInterceptor.java:59) ~[gbif-common-ws-0.23.jar:0.23]
at org.gbif.portal.action.occurrence.DownloadRequestAction.execute(DownloadRequestAction.java:54) ~[DownloadRequestAction.class:na]
at sun.reflect.GeneratedMethodAccessor1340.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_45]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_45]
{code}
The portal should be modified with a sensible display, and give people the opportunity to kill their other downloads easily - it is not currently.
This user has the following in the database:
{code}
select status, count(*) from occurrence_download where created_by = 'REMOVED_FOR_JIRA' group by status
"SUCCEEDED";27
"KILLED";13
{code}
Author: trobertson@gbif.org
Created: 2015-08-06 10:40:16.081
Updated: 2015-08-06 10:40:16.081
Looking closer, it seems the user was indeed launching many in the short space of time:
{code}
select created,status from occurrence_download where created_by='REMOVED' order by created
"2015-07-30 16:17:40.962899+02";"SUCCEEDED"
"2015-07-30 16:28:58.849525+02";"SUCCEEDED"
"2015-07-30 16:32:24.99047+02";"SUCCEEDED"
"2015-07-30 16:34:37.065238+02";"SUCCEEDED"
"2015-07-30 16:48:16.207159+02";"KILLED"
"2015-07-30 16:49:08.546164+02";"KILLED"
"2015-07-30 16:51:40.007374+02";"KILLED"
"2015-07-30 16:53:19.640247+02";"KILLED"
"2015-07-30 17:35:40.66108+02";"KILLED"
"2015-07-30 17:36:09.431292+02";"KILLED"
"2015-07-30 17:36:40.24894+02";"KILLED"
"2015-07-30 17:39:06.622105+02";"KILLED"
"2015-07-30 17:40:40.106559+02";"KILLED"
"2015-07-30 17:41:45.891958+02";"KILLED"
"2015-07-30 17:42:28.117539+02";"SUCCEEDED"
"2015-07-30 17:43:49.241773+02";"KILLED"
"2015-07-30 17:44:23.594274+02";"KILLED"
"2015-07-30 17:46:13.172275+02";"KILLED"
"2015-07-30 17:59:08.075675+02";"SUCCEEDED"
"2015-07-30 17:59:56.532515+02";"SUCCEEDED"
"2015-07-30 18:00:40.504504+02";"SUCCEEDED"
"2015-07-30 18:01:18.685903+02";"SUCCEEDED"
"2015-07-30 18:45:46.448604+02";"SUCCEEDED"
"2015-07-30 18:50:54.307123+02";"SUCCEEDED"
"2015-07-31 14:05:06.454892+02";"SUCCEEDED"
{code}
Author: ahahn@gbif.org
Comment: Download tested and succeeded, but no email notification received (worked correctly on other downloads). Reporter notified by email.
Created: 2015-08-17 11:21:51.652
Updated: 2015-08-17 11:21:51.652