How to remove orphaned tasks in Apache Mesos?

问题

The problem maybe caused by Mesos and Marathon out of sync, but the solution mentioned on GitHub doesn't work for me.

When I found the orphaned tasks:

What I do is:

restart Marathon
Marathon does not sync orphaned tasks, but start new tasks.
Orphaned tasks still took the resources, so I have to delete them.

I find all orphaned tasks under framework ef169d8a-24fc-41d1-8b0d-c67718937a48-0000,

curl -XGET `http://c196:5050/master/frameworks

shows that framework is unregistered_frameworks:

{
    "frameworks": [
        .....
    ],
    "completed_frameworks": [ ],
    "unregistered_frameworks": [
        "ef169d8a-24fc-41d1-8b0d-c67718937a48-0000",
        "ef169d8a-24fc-41d1-8b0d-c67718937a48-0000",
        "ef169d8a-24fc-41d1-8b0d-c67718937a48-0000"
    ]
}

Try to delete framework by framework ID (so that the tasks under framework would be delete too)
```
curl -XPOST http://c196:5050/master/teardown -d 'frameworkId=ef169d8a-24fc-41d1-8b0d-c67718937a48-0000'
```
but get No framework found with specified ID

So, how to delete orphaned tasks?

回答1:

There are two options

Register framework with same framework id. Do reconciliation and kill all tasks you receive. For example you can do it in following manner
- Download the code git clone https://github.com/janisz/mesos-cookbook.git
- Change dir cd mesos-cookbook/4_understanding_frameworks
- In scheduler.go change master for your URL
- If you want to mimic some other framework create /tmp/framework.json and fill it with FrameworkInfo data:
```
{
  "id": "<mesos-framewokr-id>",
  "user": "<framework-user>",
  "name": "<framework-name>",
  "failover_timeout": 3600,
  "checkpoint": true,
  "hostname": "<hostname>",
  "webui_url": "<framework-web-ui>"
}
```
- Run it go run scheduler.go scheduler.pb.go mesos.pb.go
- Get list of all tasks curl localhost:9090
- Delete task with curl -X DELETE "http://10.10.10.10:9090/?id=task_id"
Wait until failover_timeout so Mesos will delete this tasks for you.

来源：https://stackoverflow.com/questions/40098206/how-to-remove-orphaned-tasks-in-apache-mesos

标签

mesos

marathon