# Transformer's output as input to other model

**URL:** <https://discuss.huggingface.co/t/transformers-output-as-input-to-other-model/3319>\
**Category:** Intermediate\
**Created:** [January 21, 2021, 5:29am UTC](https://discuss.huggingface.co/t/transformers-output-as-input-to-other-model/3319 "2021-01-21T05:29:06Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![omerarshad](https://avatars.discourse-cdn.com/v4/letter/o/cc9497/32.png) [@omerarshad](https://discuss.huggingface.co/u/omerarshad)\
**Post date:** [January 21, 2021, 5:29am UTC](https://discuss.huggingface.co/t/transformers-output-as-input-to-other-model/3319/1 "2021-01-21T05:29:06Z")

</div>

Hello,

I want to create a model which generates text and the generated text is input to other model. So basically two models are trained together. How can i achieve this using hugging face?

Thanks

---

<div class="post-metadata">

**Author:** ![marcoabrate](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marcoabrate/32/441_2.png) [@marcoabrate](https://discuss.huggingface.co/u/marcoabrate)\
**Post date:** [January 22, 2021, 8:39am UTC](https://discuss.huggingface.co/t/transformers-output-as-input-to-other-model/3319/2 "2021-01-22T08:39:09Z")

</div>

Welcome to the forum, @omerarshad!

Nice questions, I had the same problem, too. In my opinion this is possible only if you have ground truth for the intermediate step and not only the final reference. What you might do is to train two models separately: the first one with the intermediate reference, and the last one with the final reference.

Schematically:

```
Input -> MODEL_1 -> Output_1
                        | compare (cross-entropy)
             Intermediate Reference

Intermediate Reference -> MODEL_2 -> Output_2
                                        | compare (cross-entropy)
                                  Final Reference

```

What do you think?

---

<div class="post-metadata">

**Author:** ![omerarshad](https://avatars.discourse-cdn.com/v4/letter/o/cc9497/32.png) [@omerarshad](https://discuss.huggingface.co/u/omerarshad)\
**Post date:** [January 22, 2021, 10:19am UTC](https://discuss.huggingface.co/t/transformers-output-as-input-to-other-model/3319/3 "2021-01-22T10:19:42Z")

</div>

Yes, i have ground truth for both models, and we can train them separately. The only issue i am facing is how to do using Hugging face? The task is that first model writes answer of a question, and second model takes this answer as input and generates question

---

<div class="post-metadata">

**Author:** ![marcoabrate](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/marcoabrate/32/441_2.png) [@marcoabrate](https://discuss.huggingface.co/u/marcoabrate)\
**Post date:** [January 22, 2021, 11:26am UTC](https://discuss.huggingface.co/t/transformers-output-as-input-to-other-model/3319/4 "2021-01-22T11:26:33Z")

</div>

So you have questions references and answer references (let’s call them Q\_REF and A\_REF). Then you first take the model that answers, let’s call it MODEL\_A and you train it with A\_REF. After that, you take MODEL\_Q and you train it with Q\_REF. Once you have the fine-tuned models you can just have a question as input and produce an answer with MODEL\_A, after you can take that answer and you give it to MODEL\_Q to have a new question, if I understand correctly.

You can check out the milion examples on how to train a model for Q&A with your own dataset. Or you can start reproducing results with the SQuAD one. Anyway, I never did it so I am not the best to help you on that one.

---

<div class="post-metadata">

**Author:** ![omerarshad](https://avatars.discourse-cdn.com/v4/letter/o/cc9497/32.png) [@omerarshad](https://discuss.huggingface.co/u/omerarshad)\
**Post date:** [March 27, 2021, 6:33pm UTC](https://discuss.huggingface.co/t/transformers-output-as-input-to-other-model/3319/5 "2021-03-27T18:33:40Z")

</div>

how can this be achieved using hugging face? Remember these both models will be trained together